How many tokens does your RAG stack actually send to the LLM?

The token bill for a RAG system is not set by your vector database. It's set one step later, by how you assemble the context you hand the LLM. Retrieval finds candidates; assembly decides how many of them, how much of each, and across how many LLM calls. That's where the money is spent.

So "vector DB vs. framework X vs. SWIRL" is the wrong axis. The real comparison is between context-assembly strategies. Here's an honest, source-checked look at how many tokens each common approach sends, and where SWIRL 5 actually costs less.

The honest headline first

SWIRL doesn't win by using smaller chunks. Anyone can lower top_k. On a single lean query, a minimal config like LlamaIndex's default (top_k=2) sends fewer raw tokens than SWIRL. If someone tells you SWIRL "always uses the fewest tokens," a technical evaluator will disprove it in five minutes.

SWIRL's advantage is structural, and it shows up exactly where enterprise content lives, in corpora full of document versions:

1. It never spends tokens on duplicate or superseded versions. Stock top-k returns whatever is nearest in embedding space, which in a versioned corpus means several near-identical copies. SWIRL collapses them to one canonical document before the LLM sees them.
2. It always answers in one bounded call. The multi-document synthesis modes people reach for when they want quality (LangChain refine/map_reduce, LlamaIndex refine/tree_summarize) multiply LLM calls, and refine multiplies tokens super-linearly. SWIRL never pays that tax.

What the common defaults actually do

Naive vector-DB RAG (Pinecone / Weaviate / Qdrant tutorials)

The pattern is: embed the query → retrieve top-k chunks → stuff them all into one prompt. Typical documented defaults:

• Chunk size ~512 tokens (the common "start here"; the band is 256-1024)
• Chunk overlap 10-20%
• top_k 3-5
• No de-duplication. None of the vendor quickstarts add a dedup or diversity step. Overlap alone guarantees adjacent chunks share text, and multiple versions of a document sit in the same embedding neighborhood, so top-k routinely returns redundant context and the pipeline sends all of it.

Input tokens ≈ k × chunk_tokens + overhead. At k=5 and ~1,000-token chunks, that's ~5,000 tokens, a large fraction of it redundant.

Sources: Pinecone, Weaviate, Qdrant.

LangChain

Verified from source:

• Retriever default k=4 (similarity_search(k=4) in vectorstores/base.py)
• RecursiveCharacterTextSplitter default 4,000 chars / 200 overlap (≈1,000 tokens) (text_splitters/base.py)

The combine-documents chains differ enormously in cost:

refine is the trap: each step re-transmits the accumulating answer plus the next document, so the running answer is re-sent N-1 times and grows as it goes.

LlamaIndex

Verified from source (llama-index-core):

• chunk_size 1,024 tokens, similarity_top_k 2, default response mode compact (constants.py, factory.py)
• No default de-duplication. Node postprocessors are opt-in and the default similarity cutoff is off, so two versions of a doc in top-k both go to the LLM.

Response modes (docs):

compact with top_k=2 is genuinely lean, but that's a recall trade-off, and it still sends duplicate versions.

What SWIRL 5 does (verified in source)

SWIRL treats the LLM prompt as a hard budget and fills it deliberately:

Net: ~3,000 input tokens, one call, zero redundant-version tokens, a predictable ceiling that doesn't grow with document size or corpus size.

Putting numbers on it

Scenario: a query over a corpus where the relevant policy exists in 5 versions, plus 3 other relevant documents, a realistic enterprise shape. Total LLM input tokens per query (summed across calls):

Reading it honestly:

• vs. a deliberately minimal stuff config (LlamaIndex compact, top_k=2): SWIRL is comparable per call, but that config buys its low count with poor recall and still ships duplicate versions.
• vs. typical stuff RAG (k=5, ~1k chunks): SWIRL is ~40-45% fewer input tokens, and it removes the redundant-version half entirely.
• vs. the quality-oriented multi-call modes (refine, tree_summarize, map_reduce): SWIRL is a 2-3× reduction in input tokens, and a larger reduction in output tokens, since those modes generate once per call.

Where the advantage is real, and where it isn't

• Real and hard to get off-the-shelf: cross-version de-duplication. None of these frameworks do it by default. In a versioned corpus it's the difference-maker, and it compounds: the more versions, the more SWIRL saves.
• Real: one bounded call vs. N-call synthesis; a predictable cost ceiling.
• Honest caveat: a hand-tuned vanilla RAG (low k, a good reranker, small chunks, stuff, plus your own dedup layer) can match SWIRL's per-call token count. SWIRL's value is delivering that discipline by default, and adding version de-dup the others lack. It isn't magic per-token compression.

Takeaway

If you're answering questions over a corpus with real document versioning, the tokens you waste aren't in chunk size - they're in sending the LLM the same document five times and in synthesis modes that call the model once per chunk. SWIRL 5's design removes both by default. Measure your own stack the same way: total input and output tokens per query, summed across every LLM call. That number, not top_k, is your bill.