Why your agent bills spike mid-task: the prefix cache, explained with numbers

Multi-agent runs bill in a weird shape: most requests are cheap, then one
request costs 10x its neighbors. I hit this enough times that I started
logging the JSONL usage objects, and the pattern turned out to be
prompt-cache invalidation. This post explains the mechanism with the
numbers from my own logs, and the one habit that fixed most of it.

The pricing shape

Anthropic's caching (and every provider with a similar scheme) prices
input tokens in three tiers:

cache read:   ~0.1x base input price
cache write:  ~1.25x base input price
base input:   1.0x

A stable session bills almost everything at the 0.1x rate. Your
instructions, your tool definitions, the conversation history: all of
it sits in the cache and re-reads cheap on every call. Writes happen
once, when the content first enters.

What breaks the cache

The cache works on exact prefix match. Any change to the front of the
context invalidates everything after it. In practice I have triggered
it with:

1. Editing a config file mid-run (CLAUDE.md content sits in the system block)
2. An MCP server restart that reorders tool definitions
3. Adding a tool mid-session
4. Regenerated tool_use IDs when a subagent resumes

The first three are obvious once you think about the prefix rule. The
fourth deserves a note: when a tool call is replayed after a
compaction or handoff and the IDs differ from the recorded ones, the
match fails and the whole conversation re-bills at write price.

The measurement

The JSONL transcripts under ~/.claude/projects/ carry per-request
usage. Grepping for the ratio tells the story of a session:

grep -o '"cache_read_input_tokens":[0-9]*' *.jsonl \
  | awk -F: '{s+=$2} END {print s}'

In a healthy session, cache reads dominate by orders of magnitude. In
the sessions where I saw the spike, the shape was: long run of cheap
reads, then one request where reads collapse to near zero and
input_tokens jumps to full-context size. That single request cost
more than the previous hour of work.

The fix that actually moved the number

One habit: config and tools stay frozen while a run is in flight.

Concretely:

• Rule edits batch to run boundaries, not mid-task
• Server descriptions get a monthly edit window, not ad-hoc fixes
• Long parallel work starts after config changes, not before them

This is the same discipline as database schema freezes during a
migration window, and it had the same effect: the spikes disappeared.
My worst sessions dropped from "one request costs 10x" to flat,
predictable curves.

Where I still get burned

Handoffs. When I fork a session mid-task, the resumed context
sometimes re-bills once because of regenerated IDs. I have not found a
user-side fix for that; the mitigation is to fork at clean boundaries
(after a task completes, not mid-edit), which bounds the loss to one
re-bill per boundary instead of per request.

Why this matters more for config-heavy setups

If you run a large agent config (and if you have read this far, you
probably do), your stable prefix is bigger than most people's. That is
good when it is stable: a 20k-token config reading at 0.1x is cheap.
When it breaks, you re-write 20k tokens at 1.25x on every affected
request. Big configs amplify both outcomes. Stability is what makes
size affordable.

Related reading

• Your CLAUDE.md is probably too long: keeping the stable prefix small enough to be affordable.
• I measured the token cost of 9 MCP servers: the tool-schema side of the same prefix.

Config discipline is a product for us: AgentConfig Studio ships 12 stack kits sized for a stable, affordable prefix. The Next.js sample is free (MIT).