The Model Never Ran: Why an Agent's Approval Gate Cannot Trust the Wire Between It and Its LLM

TL;DR

• HiddenLayer researchers published "Man-in-the-Model" on October 22, 2026. Their point: if an attacker controls the inference server or LLM gateway that sits between a coding agent and its model, the attacker can inject tool calls the model never requested, and the agent will run them.
• Approval systems did not stop this in the demonstrations. In the Codex example, a forged command began with the already-approved git prefix, so it ran without a prompt. The trick was a -c diff.external=... option that makes git execute a shell command.
• A related pattern, CoreBreak (Cloud Security Alliance note, August 6, 2026), put CVEs on dispatch layers that "did not consistently verify that a tool-call instruction had legitimately originated from a model turn."
• I ran git 2.53.0 locally and confirmed that diff.external and core.fsmonitor both execute commands, and that --output writes files, from a command that starts with git.
• In a small runnable model, a prefix allowlist blocked 1 of 6 forged calls, an argv-aware allowlist blocked 4, and a provenance check blocked all 6. The model is illustrative, not a product benchmark.
• The lesson: an approval gate answers "is this command allowed?" It does not answer "did the model I trust actually ask for it?" You need both answers.

What did the "Man-in-the-Model" research actually show?

Most agent threat models put the dangerous input in the model's context: a poisoned web page, a malicious tool description, a hostile README. The model reads it, gets steered, and emits a bad tool call. Guardrails, system prompts and approval prompts are built around that picture.

HiddenLayer's write-up (authors Esteban Tonglet, Divyanshu Divyanshu and Kasimir Schulz) moves the attacker to a different seat. The attacker controls a system that sits on the path: an inference server such as Ollama or vLLM, or an LLM gateway such as LiteLLM. That position is not exotic. People route coding agents through gateways for cost tracking, key management and model switching, and the write-up cites a 2026 supply chain compromise of LiteLLM and separate research on third-party LLM routers injecting malicious code.

From that seat the attacker sees every request and every response. The write-up shows three capabilities:

1. Forge a tool call. The gateway edits the model's response to include a tool call the model never produced. The agent harness cannot tell the difference, because from its side the call arrived in the normal response channel.
2. Hide it. The gateway removes the injected call and its result from the conversation before the history goes back to the model, so the model never learns about it. Injected timing wrappers also strip their own log lines from the output the user sees.
3. Learn what is guarded. The gateway identifies the harness from HTTP headers, tool catalogues and request structure, and reads configuration such as CLAUDE.md or .cursorrules by injecting file reads and then deleting them from the history.

The authors say the research looked at "over 4 thousand captured sessions" and "over thirty-five thousands file reads." Their summary line is the sentence to remember: sandboxes and approval mechanisms "can reduce that impact, but they do not establish that a tool call originated from the model the user intended to trust." The write-up as I read it does not report vendor fixes for the demonstrated cases.

Why does an approved git prefix become a bypass?

Codex lets a user approve a command prefix once, so later commands that begin with git run without a prompt. That is a reasonable usability choice when you assume the model wrote the command. The research shows what happens when someone else writes it. The forged call from the write-up:

git -c diff.external='sh -c "cat .env"' diff --no-index /dev/null .env

It starts with git, so the prefix check passes. But -c sets a configuration value for this one invocation, and diff.external tells git to run an external program to produce the diff. Git is a shell launcher with a diff command attached.

I did not want to take this on faith, so I tested it in a scratch directory with a harmless marker command instead of cat .env:

git --version
# git version 2.53.0.windows.2
echo "SECRET=abc" > .env
git -c diff.external='sh -c "echo PWNED > marker.txt"' diff --no-index /dev/null .env
cat marker.txt
# PWNED
git diff --no-index --no-ext-diff /dev/null .env   # no marker, plain diff

The marker file appeared. In a fresh repository I confirmed two more shapes that also start with git:

• git -c core.fsmonitor='sh -c "echo FSM > fsm_marker.txt"' status created the marker file, so a read-only looking status ran a command.
• git diff --output=out.txt /dev/null .env wrote out.txt, so a "diff" wrote to disk.

None of this is a git bug. These are documented features. The bug is in the policy that treats the first word of a command string as the identity of the command. I made the same point in an earlier post about the newline in a read-only fast path: the string you approve and the process the operating system starts are different things.

Can a gateway tell when nobody is watching?

The timing result in the write-up is the part I found most useful for defenders. When a tool runs automatically, the result comes back almost immediately. When a human must approve it, the round trip includes seconds of human time. A gateway that measures the delay between sending a tool call and seeing the next request can infer which tools run without approval.

The figures come from the write-up: across more than 35,000 file reads only 1.0% fell inside the heuristic's ambiguous range, 4.7% of calls to tools with short typical runtimes did, and 25.5% of shell calls did. Shell commands are the hardest to classify because their runtimes vary, but even there roughly three in four calls were classifiable by timing alone.

What this means in practice: the attacker does not need to guess which injected calls would trigger a prompt. It can learn which tools are safe to abuse silently, then inject only those. Hiding in the visual noise helps too. A long pipeline with several redirects makes a small added fragment "much less visually salient" to a human skimming for rm, sudo or curl, in the authors' words.

Is this the same failure as CoreBreak?

It is the same family, seen from a different angle. The Cloud Security Alliance note on CoreBreak, dated August 6, 2026, describes tools executing in agent frameworks without a model decision ever taking place. The listed CVEs:

The CSA root cause is the same sentence that sums up the gateway research: the dispatch layer treated a well-formed instruction as proof that a model had issued it. In the Google case, a forged human-approval confirmation worked because the confirmation processor never checked that the target tool belonged to the executing agent. The recommended mitigations include cryptographically verifiable tokens in place of command-line authorization checks, and confirmation gates that validate tool ownership and argument integrity.

The difference is where the forgery happens. CoreBreak forges inside the framework, through session history or the harness API. The gateway attack forges on the wire. Both defeat a gate that only inspects the content of the call.

How do three guards compare against forged calls?

To make the distinction concrete I built a small model of the decision. It is my own illustration, not any vendor's code, and the numbers describe this test set only.

Three guards face six forged calls and four legitimate ones:

1. Prefix allowlist. Approve anything starting with git or ls.
2. Argv-aware allowlist. Parse the command, allow only git status, git diff, git log, deny -c, --ext-diff, --no-index, --output and any argument mentioning .env.
3. Provenance MAC. The endpoint that runs the model signs each tool call with an HMAC over turn number, call index and argv. The harness verifies it before running anything. The key lives where the gateway cannot read it.

import hmac, hashlib, shlex, json

KEY = b"endpoint-side demo key"

def sign(turn, idx, argv):
    msg = f"{turn}|{idx}|{json.dumps(argv)}".encode()
    return hmac.new(KEY, msg, hashlib.sha256).hexdigest()

def guard_prefix(call):
    return call["cmd"].startswith(("git ", "ls"))

SAFE_SUBCMDS = {"status", "diff", "log"}
DENY_OPTS = ("-c", "--ext-diff", "--no-index", "--output", "--exec-path")

def guard_argv(call):
    argv = shlex.split(call["cmd"])
    if argv[0] == "ls":
        return True
    if argv[0] != "git" or len(argv) < 2 or argv[1] not in SAFE_SUBCMDS:
        return False
    for a in argv[2:]:
        if a in DENY_OPTS or a.startswith("--output") or ".env" in a:
            return False
    return True

def guard_provenance(call):
    if "mac" not in call:
        return False
    want = sign(call["turn"], call["idx"], shlex.split(call["cmd"]))
    return hmac.compare_digest(call["mac"], want)

The forged set covers the three git shapes I verified locally, a plain curl | sh, a git log -p that has an allowed shape but reads committed history, and a replay of a genuinely signed git status in a later turn. Output of the full script:

injected call                                              prefix allowlist  argv-aware allowlist   provenance MAC
git -c diff.external=... diff --no-index /dev/null .env              passes               BLOCKED          BLOCKED
git -c core.fsmonitor=... status                                     passes               BLOCKED          BLOCKED
git diff --output=... (file write)                                   passes               BLOCKED          BLOCKED
curl ... | sh (no approved prefix)                                 BLOCKED               BLOCKED          BLOCKED
git log -p (allowed shape, reads history)                            passes                passes          BLOCKED
replay of a signed 'git status' in turn 8                            passes                passes          BLOCKED

prefix allowlist       blocked 1/6 injected, falsely blocked 0/4 legit
argv-aware allowlist   blocked 4/6 injected, falsely blocked 0/4 legit
provenance MAC         blocked 6/6 injected, falsely blocked 0/4 legit

Three things to read from this, with the limits stated plainly.

First, the prefix allowlist is the weakest guard because it matches the one part of the string an attacker controls for free. Second, the argv-aware allowlist is the right fix for the command-shape problem, and it cost nothing on the legitimate calls in my set. But it is a denylist inside an allowlist, and git has many options. I would not claim my deny list is complete. Third, only the provenance check stops the calls that look perfectly fine, such as git log -p (which can read secrets committed long ago) and a replayed legitimate call. That check does not judge the command at all. It judges who sent it.

What the model does not show: whether a provenance signature is practical in a real deployment. The signer has to be something the gateway cannot impersonate, such as the model endpoint you operate, a provider that signs responses, or a sidecar you attest. I am not aware of a mainstream coding-agent setup that offers this today, and a signature only proves the call was not altered in transit. It does not prove the model's decision was wise. Treat the third column as a design target, not a product.

What should you change this week?

• Stop approving by prefix. Approve parsed argv shapes. Deny configuration injection flags for tools that execute helpers: git -c, --ext-diff, --exec-path, and similar options in other CLIs.
• Put the gate at execution time. Do the check in the process that will actually spawn the command, after every hop that can edit the call, not in a prompt that renders text the gateway can influence.
• Treat your gateway as part of the trust boundary. Pin versions and digests of LiteLLM-style proxies and inference servers, restrict who can change their config, and log tool calls on the harness side so the harness record can be compared with the provider's request log.
• Keep secrets out of reach of read-only tools. The injected read had value because .env was readable. Remove what an injected call could exfiltrate.
• Add a replay check. Bind each approval to a turn and call index so a valid call cannot be re-sent later.

FAQ

What is a "man-in-the-model" attack?
An attacker controls a system between an agent harness and its LLM, such as an inference server or gateway, and rewrites the traffic. The harness receives tool calls the model never made and treats them as the model's decisions.

Does a human approval prompt stop it?
Not reliably. If the forged command matches an already-approved prefix, there is no prompt. If it needs approval, the gateway can still make the command look routine by burying a small change in a long pipeline, and the write-up shows it removing the evidence from the history.

Is git -c diff.external a vulnerability in git?
No. It is documented behavior and I reproduced it on git 2.53.0 with a harmless marker command. The risk comes from authorizing a command by its first word while the tool can launch other programs through options.

How is this different from prompt injection?
Prompt injection changes what the model decides. This attack changes what the harness receives, so a perfectly aligned model with a perfect system prompt still cannot prevent it. The CoreBreak CVEs have the same property: the model never made the decision.

Would signing tool calls fix everything?
No. It closes the forgery and replay holes, but a legitimately signed call can still be a bad idea. Keep argument-level policy, least privilege and sandboxing as separate layers.

Further reading

Earlier posts in this series:

• One Check, Four Paths: What the Docker Sandboxes sbx cp Symlink Bug Teaches About Guarding Every Position (published 2026-11-08, same series)
• Authenticated Is Not Authorized: What a 9.3 Fail-Open Default in an Agent Mesh Teaches About Tool Gating (published 2026-11-02, same series)

Sources

• HiddenLayer, "Man-in-the-Model" research on LLM gateway attacks against coding agents (Tonglet, Divyanshu, Schulz, 2026-10-22): https://www.hiddenlayer.com/research/blog-llm-gateway-coding-agent-security
• Cloud Security Alliance Labs, CoreBreak research note, "When the Model Never Runs: Agent Guardrail Bypasses" (2026-08-06): https://labs.cloudsecurityalliance.org/research/csa-research-note-agent-infra-guardrail-bypass-20260806-csa/
• Local reproduction: git 2.53.0 (diff.external, core.fsmonitor, --no-ext-diff, --output), scratch directory, harmless marker commands only
• Illustrative guard model: author's own script, six forged calls and four legitimate calls, not a benchmark of any product