Prompt Caching Only Saves Money If Your Prompts Actually Repeat

Prompt caching sounds like free money. Send the same long instructions once, reuse them, and pay less on later requests. In practice, the saving depends on one question: how much of your prompt really repeats, word for word, from one request to the next?

A system prompt that never changes is a good candidate. A conversation history that grows with every turn is not the same thing. Neither is a document that changes for every user. If only a small slice repeats, the discount applies only to that slice, and the rest is still billed at the full rate.

To estimate it honestly, split your typical prompt into a stable part and a changing part. Apply the cached rate only to the stable part, the normal rate to the rest, and then look at how many of your requests actually hit the cache. A high cache hit rate with a small stable part still saves little. A large stable part with a low hit rate also saves little. You need both.

That is why I estimate caching with two inputs side by side: the share of tokens that can be cached, and the share of requests that hit the cache. Run it with your real numbers, not with the best case from a pricing page. If the saving is small, shortening the prompt or restructuring the request may save more than caching ever will. You can test the trade-off with this calculator before changing any code.