A stable system prompt does not guarantee a Claude prompt-cache hit

Written by Worksi's AI marketing agents and checked by a separate AI fact-checking agent. The original is at https://blog.worksi.ai/claude-prompt-cache-miss-same-system-prompt.

You diffed the system prompt. It is byte for byte the same on every request. Yet the usage on each response shows a fresh cache_creation_input_tokens and a cache_read_input_tokens of 0, and the input bill can be higher than it would be with caching off, because writes cost more than plain input. A Claude prompt cache miss despite the same system prompt is not a contradiction: the system prompt is not the cache key. The key is the whole prefix up to the block that carries cache_control, and that prefix starts before the system prompt and often ends after it.

So stop asking "did the system prompt change?" and ask four narrower questions, each of which a pair of controlled requests can answer: did anything before it change, did the breakpoint include something that changes, did the entry expire, and was the second request sent before the entry existed. This article gives a test design for that. It is a proposed diagnostic built from Anthropic's current documentation. We have not run it for this article, so every expected result below is a hypothesis.

The cache key is the whole prefix up to the breakpoint.

Anthropic's prompt caching guide states the order: caching "covers the full prefix: tools, system, and messages (in that order), up to and including the block designated with cache_control." Three rules from the same page decide whether a request hits:

• Writes happen only at a breakpoint. A marked block writes exactly one entry, a hash of the prefix ending at that block. "The system does not write entries for any earlier position."
• Reads look backward for earlier writes, not for stable content. If the hash at the breakpoint has no entry, the system walks back one block at a time, at most 20 positions, looking for an entry a prior request wrote.
• Entries appear only once a response starts. The prefix is cached "once the response starts", and for parallel requests the guide says to "wait for the first response before sending subsequent requests."

Put those together and an identical system prompt can miss in at least four ways:

1. A tool definition changed. Tools come first. The guide's invalidation table says modifying tool names, descriptions or parameters "invalidates the entire cache", system and messages included. A tool list built from an unordered collection, or a description that embeds a version string, is a candidate.
2. The breakpoint sits on a block that changes. The guide's own "common mistake" is a breakpoint on a block holding a timestamp and the user message. Each request writes a new entry at that block, the lookback finds no earlier write behind it, and "you pay for a fresh cache write on every request and never get a read." Automatic caching, the top-level cache_control field, has the same problem when the last cacheable block is the one that varies.
3. The entry expired. The default lifetime is 5 minutes, or 1 hour at a higher write price. It is measured "from the start of the request that writes or reads the cache entry, not from the end of its response", so a 4-minute stream leaves about a minute for the follow-up.
4. The second request went out too early. Parallel requests sent before the first response starts find no entry, and each can write its own.

Two outcomes look like misses but are not mismatches. A prefix below the model's minimum is "processed without caching, and no error is returned"; both cache fields are then 0. On the Claude API, the guide lists, among others, 1,024 tokens for Claude Sonnet 4.6 and 4,096 for Claude Haiku 4.5 and Claude Opus 4.6. Check the current table for your model and platform. And a growing conversation can push the breakpoint 20 or more blocks past the last write, out of the lookback window. This article covers the Claude API; Amazon Bedrock documents its own minimums and usage-field names.

Log fingerprints and usage, not prompts.

You need to compare prefixes across requests without writing prompt text to logs. Fingerprint each block locally, in prefix order, and keep a running prefix fingerprint that also covers each block's section, message role and position, so the first divergence is easy to find.

import hashlib
import hmac
import json

FP_KEY = b"load-from-your-secret-store"  # keyed, so short blocks cannot be guessed from logs

def _digest(data: str) -> str:
    return hmac.new(FP_KEY, data.encode("utf-8"), hashlib.sha256).hexdigest()[:16]

def _fingerprints(block: dict) -> tuple[str, str]:
    body = {k: v for k, v in block.items() if k != "cache_control"}
    canonical = json.dumps(body, sort_keys=True, separators=(",", ":"), ensure_ascii=False)
    as_built = json.dumps(body, separators=(",", ":"), ensure_ascii=False)
    return _digest(canonical), _digest(as_built)

def prefix_rows(tools, system, messages):
    """One row per block, in cache prefix order: tools, system, messages."""
    if isinstance(system, str):
        system = [{"type": "text", "text": system}]
    blocks = [("tools", None, i, t) for i, t in enumerate(tools or [])]
    blocks += [("system", None, i, b) for i, b in enumerate(system or [])]
    for m, msg in enumerate(messages):
        content = msg["content"]
        if isinstance(content, str):
            content = [{"type": "text", "text": content}]
        blocks += [(f"messages[{m}]", msg["role"], i, b) for i, b in enumerate(content)]

    rows, running = [], ""
    for section, role, index, block in blocks:
        canonical, as_built = _fingerprints(block)
        # structure goes into the running digest: section, role and position
        marker = json.dumps([section, role, index], separators=(",", ":"))
        running = _digest(running + "|" + marker + "|" + canonical)
        rows.append({
            "section": section,
            "role": role,
            "index": index,
            "fp": canonical,
            "fp_as_built": as_built,
            "prefix_fp": running,
            "breakpoint": "cache_control" in block,
        })
    return rows

The fingerprints answer different questions. fp is content-only: it covers one block's body and nothing about where that block sits. prefix_fp adds structure: a changed role, a block moved into another message, or a block inserted earlier changes it, even when every fp is the same. If fp matches between two requests but fp_as_built does not, your code built the same content with a different key order. Whether the provider's hash is sensitive to that is not documented in the pages cited here, so treat it as a candidate to eliminate, not a cause. If fp itself differs, the content changed.

The function sees only tools, system and messages. Request parameters outside them, and the model id, are not in any fingerprint: record those and compare them separately.

These fingerprints are diagnostic only. Even complete local fingerprints are not the provider's cache key and cannot be: the SDK serializes the request its own way, and the API may add content you never see. The guide's invalidation table, for example, says enabling or disabling web search modifies the system prompt. Matching local fingerprints make a hit more plausible. They do not guarantee one.

For every request, record:

• the model id, the API endpoint, the organization or workspace, and a non-secret credential alias or key ID, so pairs are comparable; never log the API key itself
• the other request parameters you send besides tools, system and messages
• the request ID your client exposes, and the response's message id
• sent_at, and first_event_at (the first streamed event); with a non-streaming call you only know when the whole response arrived, which is later than when it began
• breakpoint positions and the TTL you requested, or "automatic" for top-level cache_control
• prefix_fp at each breakpoint
• input_tokens, cache_creation_input_tokens, cache_read_input_tokens, stored as null when the response did not report them

A paired-request test matrix.

Each row is a pair or group of requests that differ in one variable. Use the same model, API, organization, request parameters, prefix length and breakpoint placement unless the row says otherwise. Put a fresh random nonce in the first system block of each test run, so the run's entries cannot collide with entries your production traffic wrote for the same prefix. Use a prefix comfortably above the model's minimum, except in row G.

This is a synthetic test design. The expected observations are hypotheses that hold only if the request is eligible, the provider behaves as its documentation says, and that documentation has not changed.

Row E needs the entry left alone: any request that reads it refreshes the lifetime "for no additional cost", so a stray retry turns an expiry test into a hit. Row F does not tell you how many writes to expect, only that the documentation gives no reason to expect reads. If B fails to hit, stop: nothing else in the matrix is interpretable until the baseline pair behaves as documented.

Count input in three fields before you price it.

The rate limits page defines the total:

total_input_tokens = cache_read_input_tokens + cache_creation_input_tokens + input_tokens

input_tokens is only the part "after your last cache breakpoint", so a cost report that multiplies input_tokens by the input price undercounts every cached request. The same page says cache reads do not count toward input-tokens-per-minute rate limits. That is a rate-limit rule. Reads are still billed.

Each category has its own price. The caching guide lists 5-minute writes at 1.25 times the base input price, 1-hour writes at 2 times, and reads at 0.1 times, with per-model exceptions at 0.05 and 0.025 times. These multipliers stack with other modifiers such as the Batch API discount, long-context pricing and data residency, so read the rates for your model from the current pricing table instead of hard-coding one.

A fictional ledger, in base-input-price units, 5-minute TTL, a model on the standard 0.1 read multiplier:

Multiply weighted units by your model's base input price per token. Row D1 is the expensive case: a write that no request ever reads costs more than sending the same tokens uncached. The last row stays unknown. Writing 0 there would make the cache look better or worse than it is. If you also enforce per-user limits, whether cache reads count toward a user's quota is a separate policy choice, covered in what a per-user token quota should count.

A decision tree for one suspicious miss.

Start from a request whose usage shows a write where you expected a read.

1. Was usage reported? If not, the cache state of that request is unknown. Stop there.
2. Are both cache fields 0? Then nothing was cached or read. Check that cache_control was present, that the prefix met the model's minimum on that platform, and that automatic caching found an eligible block. This is not a prefix mismatch.
3. Do the model id, endpoint, organization or workspace, and the other request parameters match the writer's? If not, the requests are not comparable. Rerun the pair with those held fixed.
4. Did the request start after an earlier request's first event? If it started before, concurrency is a candidate. Repeat the pair sequentially.
5. Did it start within the TTL of the start of the last request that read or wrote the entry? If not, expiry is a candidate. Note which TTL that writer requested.
6. Where does prefix_fp first diverge from the writer's? In tools, a tool change. In system, a system change your diff missed. In messages before or at the breakpoint, the breakpoint includes a changing suffix: move it to the last stable block and rerun the D2 pair. If fp matches at the first divergent row, the role, section or position changed, not the content.
7. Is the breakpoint 20 or more blocks past the writer's breakpoint? Then the lookback window is a candidate. A second breakpoint closer to the last write is the guide's remedy.
8. Everything matches and the timing is right? The result is inconclusive. Candidates include serialization your fingerprint does not see, content the API adds, and provider behavior the documentation does not describe. Rerun the pair in isolation with a fresh nonce before drawing a conclusion.

A partial read, where cache_read_input_tokens is above 0 but smaller than you expected, fits the same pattern as step 6: the lookback may have found an earlier entry, with something after it changed. Usage alone cannot tell you which of these happened. It only tells you what was written and read. Name a cause only when a controlled pair reproduces it.

Where a gateway fits.

Everything above needs only your own request log and the three usage fields. If your Claude traffic passes through a gateway, check that the gateway's cost records price all three fields separately before you trust its totals. Worksi AI Control is one such gateway: it traces each model call and prices it from your model registry. The paired-request test still compares the request you send with the usage the provider returns.

A test run costs real money: each forced write in rows A, C, D1, E and F is billed at the write rate. The design cannot see the provider's real cache key, and it assumes the documentation describes current behavior. Model-specific minimums and prices change, so recheck the guide before you run it.

Originally published at blog.worksi.ai.