How a KV cache bug exposed other customers' prompts

For about two weeks this summer, a small number of people using Cloudflare's GLM-5.3 model on Workers AI got answers stitched partly from someone else's prompt. Nobody stole a password. A memory bookkeeping mistake in the KV cache let 155 responses read pages a previous customer had left behind. Cloudflare published a coordinated disclosure on October 26. The leak window ran from August 27 to September 9, 2026, and only prompts longer than 681,280 tokens were affected.

What is a KV cache in an LLM?

A language model writes one token at a time. To pick each one, attention looks back over every earlier token, and for each of them it needs a key and a value. Recomputing those for the whole conversation on every step would be wasteful, so the model stores them. That store is the KV cache. It grows with context length times the number of people served at once, and it can end up bigger than the model's weights.

Serving engines such as SGLang do not give each request one giant block of memory. They carve GPU memory into a shared pool of fixed-size pages and lend pages out. When a request finishes, its pages go back to the pool without being wiped, because overwriting is the next request's job.

The two numbers that did not match

According to Cloudflare's launch changelog, GLM-5.3 shipped with a 1M-token context window. The advertised figure was 1,048,576 tokens, but the pool holding the prefill KV cache fit only 681,280. Any prompt above that should have been refused.

The setup was disaggregated: prefill nodes read the prompt and build its cache, decode nodes take that cache and generate the reply. Cloudflare's root-cause analysis describes this chain:

1. A prompt longer than the pool arrives.
2. SGLang marks it rejected and swaps in a one-token placeholder instead of failing loudly.
3. The admission check only asks whether a request is finished. A rejected request counts as finished, so it passes.
4. The decode node reserves 8 KV pages, expecting a full handoff. Prefill wrote one.
5. Attention then runs over all 8 pages, and seven still hold keys and values from earlier requests to the same model.

The reply was fluent and came back with HTTP 200, so nothing looked wrong on either side. Cloudflare says its pre-launch tests checked status codes, not what the model actually said.

Keeping it in proportion

Cloudflare counts 155 affected responses, all from very long prompts, on one model, and says it saw no evidence of deliberate exploitation. The same SGLang flaw had been reported upstream as issue #30233 on July 6, and a fix was merged on September 6 in SGLang v0.5.20, three days before the incident was declared. The prefill pool now holds about 2.04 million tokens, roughly twice the context length.

KV cache security: what to check if you run inference

Cloudflare's advice is specific:

• After any change to model weights, quantization, parallelism or memory settings, compare the KV pool size with the advertised context length on every prefill and decode node.
• Upgrade SGLang to v0.5.22 or later. v0.5.21 added a follow-up fix.
• Test what the model returns, not only the status code. Send an over-length prompt and confirm it is refused.

If you only call shared APIs, treat any multi-tenant service as a place where a bug could, rarely, expose what you send. Keep secrets and regulated personal data out of prompts unless a contract and risk review cover them.

The pool size is also a security boundary. Every trick that squeezes more tokens into a fixed pool, such as KV cache quantization or offloading pages to CPU memory, changes how many tokens fit inside it.

The original has the full timeline of September 9 and a short FAQ: https://syntaxradar.com/radar/kv-cache-explained-cloudflare-prompt-leak