How a GPU Memory Mismatch Let Cloudflare's AI Serve One Customer Another's Data

https://rohantechofficial.com/tech-news/cloudflare-workers-ai-glm-data-leak/

Cloudflare's October 26 incident report is worth reading in full, because the bug is more interesting — and more instructive — than the headline. For about two weeks (Aug 27–Sep 9), 155 responses from the GLM-5.3 model on Workers AI contained fragments of other customers' requests. All 155 went to prompts longer than 681,280 tokens. Nobody hacked anything.

Two things went wrong at once. First, the model advertised a ~1M-token context window, but the GPUs' KV cache pool only fit 681,280 tokens — SGLang computes that pool from leftover GPU memory at startup, and nothing checked it against the advertised length. Second, when an over-length prompt arrived, SGLang's disaggregated prefill/decode path marked it rejected but didn't abort it; the decode node ended up generating from stale cache pages holding previous requests' data.

The response was textbook: researcher Gadi Evron reported via HackerOne on Sep 8; Cloudflare reproduced it within 25 minutes of declaring an incident on Sep 9, pulled the model within 40 minutes, rolled the fix to all nodes, and restored service the same evening. Then it worked with SGLang maintainers on upstream fixes (v0.5.21+) before publishing the full postmortem naming its own errors.

Takeaways: if you sent 681K+ token prompts to GLM-5.3 in that window, ask Cloudflare support about your account. If you self-host SGLang with disaggregated serving, upgrade past v0.5.22. And remember — advertised context length and actual GPU memory are two different numbers. Verify the second one.

Full technical breakdown: https://rohantechofficial.com/tech-news/cloudflare-workers-ai-glm-data-leak/