The State of Agent Code Safety: what we scanned, and what we refused to flag

The one-sentence problem

A static analyzer sees eval(x) and asks "is x tainted by SQL or an HTTP parameter?" It has no concept of "x is the model's reply."

That blind spot — model output reaching a code, shell, or deserialization sink — is the entire agent layer, and it's invisible to the tools most teams already run.

I scanned 30+ of the most-starred open-source AI-agent frameworks for exactly this, then hand-verified every high-severity finding against the source. Here's what's real — and, deliberately, what I decided not to report.

What's actually out there

~1 in 5 frameworks let model output flow into eval/exec/new Function — mostly code-writing agents by design, where the only thing between a prompt injection and RCE is a sandbox the scanner can't verify.
One is unambiguous and public: SuperAGI's eval() on the assistant's own reply, CVE-2025-51472.

73% assemble LLM request params in a dict, spread them with **kwargs, and set no max_tokens anywhere in the path (27 call sites).

Cost hygiene, not a vuln, on its own — but near-universal, and passed indirectly where most scanners can't see it.

The part nobody publishes: what I refused to flag

A scanner that cries wolf gets demoted to advisory and ignored. The hardest — and most valuable — engineering here is not flagging the things that look scary but aren't. Every one of these is a real pattern in a real, well-engineered framework that a lazy scanner reports as critical:

The rule that produces that discipline: assert HIGH only when you can see the dangerous input's source — a value assigned from an LLM call in scope, or an unambiguous name like request.body.

When the danger is only inferred from a variable's name, say so — MEDIUM, "confirm the source" — not "you have an RCE." A judge rules on the evidence in front of it.

What I'm not claiming

No precision/recall number yet — that needs a labeled benchmark with true negatives, which I'm building. This is a methodology plus hand-checked findings, plus a public account of the false positives I eliminated.

Honesty about the second is the point. (One reported finding — an unbounded tool-call loop in LightAgent — already got a maintainer fix. That's the bar.)

Read the full report / scan your own agent

Full write-up: https://release-gate.com/research.html

pip install release-gate

release-gate audit . # two honest scores + a PROMOTE/HOLD/BLOCK verdict

It's open source — the analyzer and the entire false-positive test suite are on GitHub.(https://github.com/VamsiSudhakaran1/release-gate)