CVE Remediation Agents: What Goes in the Context Pack Before a Machine Fixes 47 CVEs

The second most-read story on daily.dev right now is a claim that stops most engineers mid-scroll: an AI agent was given the right context and fixed 47 CVEs across PHP, Node.js, Python and Go — 127 upvotes, 1.2M views, and the hottest security-and-automation conversation in the feed. It sits above stories about Arc shutting down and the terminal-as-teacher essay, and right next to a PHP2Shell patch guide that is itself still trending. The question the headline provokes is the wrong one. Nobody serious doubts an agent can rewrite a package.json version pin. The interesting question is what "context" meant in that sentence: what the agent was allowed to see, what it was not allowed to decide, and — critically — what verified that the 47 fixes were real and not 47 new problems.

We run automation that touches production systems: emergency patch rollouts across fleets, agent-driven workflows, unattended pipelines. Our PHP2Shell patch-operations playbook covered the execution side — how you roll a fix across many machines safely. This piece is about the input side, which is where agent remediation actually lives or dies: assembling the context pack an agent needs to remediate vulnerabilities without supervision, and the verification loop that makes the fixes trustworthy. It follows the same philosophy as how we work — automate aggressively, and spend human judgment where it changes the outcome.

1. The bottleneck is context, not capability

Give a modern coding agent a CVE number and repository access and it will almost always do something. That is the problem. Left to its own devices, an agent:

• upgrades the wrong thing (bumps a transitive dependency's direct cousin, or "fixes" a devDependency that never ships),
• reads the advisory title but not the affected versions field, patching a version range that does not include your pinned one,
• applies the upstream fix commit to a codebase that has diverged six months past the tagged release,
• or "fixes" the CVE by deleting the code path — technically eliminating the vulnerability, practically deleting a feature.

Every one of these failures is a context failure, not a capability failure. The model can reason about the fix; what it cannot do is conjure the four facts that determine whether a fix is correct: what exactly is installed, what is actually reachable, what the fix touches, and how you will know it worked. A remediation pipeline that works treats those four facts as a data-assembly problem, not a prompting problem.

2. The context pack: what an agent needs before it touches anything

Before any agent session begins, build a single document — the context pack — with these sections. We generate ours from tooling, never by hand, because hand-written context rots within a week.

2.1 Precise inventory. Not "we use Django" — the exact resolved dependency tree with versions, drawn from the lockfile (package-lock.json, poetry.lock, go.sum, composer.lock), plus the runtime inventory: which interpreters and system packages are actually deployed. Lockfile versions and deployed versions disagree more often than anyone admits; the deployed one is what matters.

2.2 The advisory, normalized. For each CVE: the affected product, the affected version ranges in machine-readable form, the weakness type (CWE), the attack vector and required privileges. Sources: the NVD, the GitHub Advisory Database, and osv.dev, which exposes a clean JSON API for exactly this and is the easiest source to script against. Normalize into one schema; every source formats ranges slightly differently, and an agent reading raw advisory prose will mis-parse version ranges about as often as a tired human.

2.3 Reachability analysis. This is the section most pipelines skip and the one that matters most. A vulnerable function that is never called, in a module that is never imported, in a container that never runs, is not your Tuesday emergency. Compute: does any code path you ship reach the vulnerable symbol? Tools like npm ls, pipdeptree, go mod why, and call-graph analysis get you most of the way. Mark each finding reachable / unreachable / unknown — and treat "unknown" as reachable until proven otherwise.

2.4 The fix options and their blast radius. For each finding, enumerate the actual options: upgrade to a patched release, backport the upstream patch, apply a configuration mitigation (disable the feature, tighten the firewall rule), or accept the risk with a documented reason. For each option, list what else it touches: breaking changes in the target release, config format changes, migrations. The agent's job is to pick among enumerated options with stated trade-offs, not to invent options.

2.5 Policy and provenance. Who owns this service. Whether it is internet-facing. What data it touches. And a provenance rule borrowed from our agent-git workflow: every agent commit must be attributable and revertible — see our Git for AI agent teams piece for the full discipline. The agent needs to know the blast radius of its own work, and the humans need to audit it later.

A context pack is typically 200–600 lines of structured data for a medium service. That sounds like a lot to assemble; it is the entire job. Everything after this is comparatively mechanical.

3. Triage before fixing: CVSS scores lie, exploitability decides

The original story says "47 CVEs." A raw count of CVEs is the least informative number in vulnerability management. Here is the triage order that actually works, and the order you should encode in the agent's instructions:

1. Known-exploited first. If the CVE is in CISA's Known Exploited Vulnerabilities catalog, it jumps the queue — exploitation in the wild is the only signal that beats theory.

2. Reachable + network-exposed second. Reachable vulnerable code on an internet-facing service.

3. Reachable + internal third. Then unreachable, then dev-only.

Two signals sharpen this beyond CVSS, which measures severity in a vacuum:

• EPSS (Exploit Prediction Scoring System) estimates the probability of exploitation in the next 30 days. A CVSS 9.8 with EPSS 0.01 is a different animal from a CVSS 7.5 with EPSS 0.6.
• Attack complexity and privileges required, from the CVSS vector string itself — an agent can read AV:N/PR:N/UI:N and correctly conclude "no user interaction, no privileges, network vector: worst case."

The practical rule we give agents: fix in exploitability order, verify in the same order, and say out loud which findings you are deferring and why. An agent that silently fixes 47 low-risk findings while the one KEV-listed RCE waits is worse than an agent that fixed nothing — it manufactured confidence.

4. The fix loop: reproduce → patch → retest → regress

A remediation without a verification step is a rumor. Every fix the agent makes must pass this loop, and the loop — not the agent's self-assessment — is what declares a CVE fixed:

1. Reproduce (or prove unreproducible). Where feasible, demonstrate the vulnerable behavior first: a failing security test, a proof-of-concept input, or at minimum the scanner output showing the finding. This is the "before" snapshot.

2. Apply the minimal fix. Prefer the vendor's patched release over hand-rolled patches. Prefer the smallest version bump that clears the affected range — jumping three major versions to fix a CVE trades one risk for a regression lottery.

3. Retest the finding. Re-run the scanner or the security test. The finding must be gone for the right reason — the version is out of the affected range, not because the scanner config changed.

4. Regression-test the neighborhood. Run the service's own test suite, at minimum the tests covering the changed dependency's integration points. A fix that breaks checkout is not a fix.

5. Diff review. A human or a strict policy gate reviews the final diff: dependency changes only, no drive-by refactors, lockfile consistent with the manifest.

Step 5 is the human gate, and it stays human. Which brings us to what the agent never decides alone.

5. What we never let the agent decide alone

Some decisions carry blast radius that no context pack can fully capture. Our hard rules:

• Network-exposed services get human sign-off on the diff. The agent can prepare everything; a person approves the merge. The cost of a 30-second review is nothing next to a bad deploy on the front door.
• Data-bearing changes are human decisions. Anything touching authentication, encryption, PII handling, or payment code: the agent proposes, a human disposes. This is the same station-gating logic from our dark software factories analysis — the factory runs dark at the safe stations and has lights at exactly the stations where darkness gets expensive.
• Risk acceptances are human, written, and expiring. "We accept this CVE because the code path is unreachable" is a fine decision — made by a person, recorded with the evidence, with an expiry date. Agents do not get to accept risk; they get to gather the evidence for the acceptance.
• Rollback must exist before rollout. Same rule as the PHP2Shell playbook: no fix ships without a tested path back.

Notice the pattern: the agent does the doing (triage, patching, testing) and separate mechanisms — scanners, test suites, human gates — do the verifying. The moment the doer and the verifier are the same mind, you have optimism, not remediation.

6. Failure modes we have actually seen

A short catalog, because knowing the failure modes is half the defense:

• Context bloat. Feeding the agent the entire advisory database "just in case." Past a point, more context degrades decisions — the agent starts pattern-matching to the wrong CVE. Curate: only the findings that affect this service, only the fields in section 2.
• Hallucinated versions. The agent asserts a patched release exists that does not, or confuses the fixed version across ecosystems ("lodash 4.17.21" is real; the agent's memory of which release fixed what is not). Ground every version claim in a live registry or advisory lookup.
• The silent no-fix. The scanner goes green because the agent pinned the dependency to a version outside the scanner's database or excluded the path from scanning. The retest step (section 4.3) must check why the finding disappeared.
• Supply-chain whack-a-mole. Fixing a direct dependency while the same vulnerable code arrives via a transitive path. The inventory in section 2.1 must be the full resolved tree, and the retest must scan the tree, not the manifest.
• Fixing the scanner's opinion, not the vulnerability. Upgrading past the affected range per the advisory while the actual vulnerable code was vendored/copied into the repo years ago. Reachability analysis on vendored code is manual work — flag it as unknown and let a human look.

7. A minimal pipeline you can build this week

You do not need a platform. The smallest useful version is two scripts and a policy file:

Script 1 — context pack builder. Walks the lockfile, queries the OSV API for each pinned version, joins the results with a simple reachability check (import graph for Python, npm ls for Node), and emits the context pack as JSON. The OSV API is public, unauthenticated, and takes batch queries — this is an afternoon's work, and it is the highest-leverage afternoon in the whole pipeline.

Script 2 — verification runner. Takes a list of finding IDs, re-runs the scanner, runs the affected service's test subset, and reports pass/fail per finding with the reason. This script is the verifier; it must be deterministic and boring.

Policy file — the agent's standing orders. Triage order (KEV → reachable/exposed → reachable/internal → rest), the minimal-fix preference, the mandatory retest loop, and the list of decisions that require a human. This file is short and changes rarely; it is the constitution the agent operates under.

Run it on one service first. Measure two numbers: median time from advisory to verified fix, and the false-fix rate (fixes that had to be reverted or re-done). If the second number is not near zero, your context pack is incomplete — go back to section 2, not to a bigger model.

Frequently asked questions

Does the agent need the full CVE database in context?
No. It needs the findings that affect its service, normalized, with reachability attached. The full database is a lookup tool the agent queries, not context it carries.

CVSS, EPSS, KEV — which one do I sort by?
KEV membership first (exploited in the wild beats everything), then EPSS-weighted reachability for the rest. CVSS alone is severity without likelihood; it is the weakest of the three for prioritization.

Can an agent really do the reachability analysis?
Partially. Import graphs and dependency trees are automatable and reliable. Full call-graph reachability across dynamic languages has false negatives — mark those unknown, treat as reachable, and let the human gate handle them.

What is the single most common mistake?
Skipping the "why did the finding disappear" check. A green scanner is not a fixed vulnerability; it is an absence of a signal, and absences have causes. Verify the cause.

How is this different from just running Dependabot?
Dependabot opens version-bump PRs — valuable, but it does not do reachability triage, does not weigh EPSS, does not verify beyond CI, and does not explain deferrals. The context-pack pipeline is the judgment layer around the same mechanical upgrades.

References

• OSV — distributed vulnerability database and API: https://osv.dev/
• CISA Known Exploited Vulnerabilities Catalog: https://www.cisa.gov/known-exploited-vulnerabilities-catalog
• FIRST EPSS (Exploit Prediction Scoring System): https://www.first.org/epss/
• NVD — National Vulnerability Database: https://nvd.nist.gov/
• GitHub Advisory Database: https://github.com/advisories
• daily.dev Popular feed (trend signal for this piece): https://daily.dev/posts?sortBy=popular

Originally published on botslave.com. Botslave is an automation studio — see what we build and how we work.