Cloudflare D1’s 50 MB Cap Exposes a Trap in Paginated Search

A support archive can show 25 case cards and still pull thousands of work notes to render them. That difference matters on Cloudflare D1. The company announced on October 30 that it would start rejecting query results larger than 50 MB on November 4, 2026. D1’s limits page lists the cap and exempts REST API import and export operations. The first thing to check is what the query returns before your application groups, trims, or renders it. Cloudflare announcement · D1 limits

Disclosure: AI generated and fact-checked this article against the linked public sources. Keith has not reviewed it.

The failure can hide behind a reassuring page size. Imagine a support dashboard that selects 25 parent cases in a subquery and joins all their notes in the outer query. If those cases each have 200 notes averaging 12 KB of text, the note bodies alone total about 60 MB before accounting for other columns. Those are invented fixture values, and the arithmetic describes the text alone. Cloudflare’s own accounting determines whether a real response crosses its cap.

Limiting the outer join to 25 rows cuts the result, but it changes what a page means. A single case with 25 matching notes could occupy the whole page. If the interface promises 25 cases, the query needs to preserve the parent case as the unit of pagination.

Page the case IDs first

The public Illumia Archive Intelligence case study describes a local PostgreSQL application that groups search results by parent case and highlights matching work records in the case history. The implementation was tested with synthetic records, with workplace acceptance staged separately. The following design uses that interface as a reference for a hypothetical D1 implementation. Illumia case study

Start with a page query that returns one row per matching case. Use an existence check when the filter only needs to know whether a matching note exists. Select the case ID and the small fields the card actually displays. If cards need a note preview, fetch a bounded set of excerpts for the visible cases, keeping the case ID on each excerpt. A count of two excerpts per case still needs a length bound. A short preview can carry a note ID so that the full context remains one deliberate click away.

Give the page a deterministic order. For an ascending list ordered by a non-null creation timestamp and a unique case ID, the next page starts after the last timestamp and ID pair. Keep that ordering and its cursor consistent with the current filters. If the filter changes, restart the page. SQLite documents this approach and explains why large offsets can require it to work through rows it never returns. An index that matches the access pattern matters, and a cursor does not freeze a changing dataset. SQLite’s scrolling window example

There’s a tradeoff. Splitting the card and detail reads may cost an extra round trip, and an existence check on a poorly indexed filter can still be expensive. Look at the query plan and measure the bytes for each read separately. A small response can still take too long to produce. Loading the first page also should not trigger 25 separate requests for complete histories.

Make one case ruin the average

Try a synthetic fixture with many ordinary cases and one case with hundreds of long notes. Include two cases that share a sort timestamp, and a search term that appears in several notes under the same case. Check that each case appears once in the list, that page boundaries retain both cases with tied timestamps, and that opening a matching note still lands inside the right history. Measure the list, excerpt, and detail responses separately. The detail view may need its own pagination when the same unusually large case opens.

Run the fixture through the application’s error path too. If the query fails, preserve the search terms and show that the results could not load. A successful search with zero matches tells the operator something different. Keep administrative exports on an explicitly authorized path designed for bulk data, with their own size and completion checks.

Support teams often open an archive because they need the context behind one earlier incident. The search page has done its job when they can find that case and follow the right note into its history, even when another case on the same page has years of entries.