Naive RAG is lying to you. Here are the 7 architectures that actually survive production.

The first RAG pipeline everyone builds is the same: chunk the docs, embed them, retrieve the top few by cosine similarity, stuff them in the prompt, let the model answer. It demos beautifully. The answers sound right. You ship it.

Then someone asks a question that actually matters, and it pulls the wrong chunk. Or it finds the right document and misses the one paragraph you needed. Or the answer is sitting in your database and the model never sees it.

Nothing is broken. The pipeline is doing exactly what you built it to do. That is the problem.

What you built is Naive RAG, and it isn't useless, it's where you start. But getting documents into a vector database is the easy half. Getting the right information into the model's context window is the real job, and Naive RAG barely attempts it.

Here's the reframe that makes this whole post useful: there is no single "advanced RAG." There are seven distinct architectures, and each exists because retrieval fails in a specific way. Pick the wrong one and you add cost and latency for nothing. So before the list, learn to name the failure.

The five ways retrieval actually fails

1. Vocabulary mismatch. User types "how do I cancel," the doc says "terminate your subscription." Semantic search helps, but exact terms, product codes and names still get missed.
2. Lost context. A chunk says "revenue grew 3%." Whose? Which quarter? The embedding has no idea, because the chunk was torn out of its document.
3. The question isn't the answer. A short question embeds nowhere near the long, technical passage that answers it. The vectors just don't line up.
4. The answer is the whole document. "Summarize the themes across these 400 reports" can't be answered by any 5 chunks. No amount of top-k reaches it.
5. Retrieved, then wrong anyway. The system pulls plausible-looking chunks that don't contain the answer, and the model confidently writes from them.

Each architecture below targets one or two of these. Match them, and you stop cargo-culting complexity you don't need.

1. Hybrid Search + Reranking

Fixes: vocabulary mismatch

This is the upgrade everyone should make before anything fancier, and most people skip it to chase agents.

Pure vector search is semantic: great at "cancel" ≈ "terminate," terrible at exact matches like an error code or a rare surname. Hybrid search runs a dense vector search and a classic keyword search (BM25) and fuses the results, usually with Reciprocal Rank Fusion, which scores each doc by summing 1 / (k + rank) across the lists it appears in. Dense catches meaning; BM25 catches the literal token the embedding smoothed away.

Then you rerank. Retrieval is tuned for speed, so it over-fetches (say 150 candidates), and a slower cross-encoder re-scores those against the query and keeps the best 20. The cross-encoder reads query and document together, so it catches relevance a bi-encoder embedding misses.

When to use: always. This is table stakes, not an advanced technique. If your RAG has no reranker, fix that before reading on.

2. Query Transformation

Fixes: the question isn't the answer

Your user writes a short, messy question that embeds nowhere near the answer. So you rewrite the question before you search. Four flavors:

• Multi-query: ask the LLM to rephrase the question 3-4 ways, retrieve for all, merge. Catches the angle the original missed.
• RAG-Fusion: multi-query plus Reciprocal Rank Fusion on the combined results. Docs that surface across several rephrasings float up. Caveat from the original work: if the generated queries drift off-topic, so do the answers.
• HyDE (Hypothetical Document Embeddings): instead of embedding the question, ask the LLM to write a fake answer, then embed and search with that. A hypothetical answer looks far more like the real document than the question does. It can hallucinate a bad hypothesis, so pair it with reranking.
• Step-back prompting: for a narrow question, first ask a more general one ("what are the principles behind X?"), retrieve the background, then answer the specific.

When to use: when the right docs exist but the query doesn't find them. Cheap, one extra LLM call.

3. Contextual Retrieval

Fixes: lost context

When you chunk a document, each chunk loses the story around it. "Revenue grew 3%" is useless without "ACME Corp, Q2 2026." Anthropic's Contextual Retrieval prepends a short, LLM-generated blurb to each chunk that situates it in its document before embedding, so the vector and the keyword index both carry the context.

This is the one architecture here with clean published numbers. Measuring the share of queries where the right chunk wasn't in the top 20:

Cost is roughly $1.02 per million document tokens, one-time, using prompt caching. One honest caveat: those figures are from the best-performing embedding model in Anthropic's test, not an average. Still, 5.7% → 1.9% from a preprocessing step is one of the best returns in this whole list.

4. Hierarchical Retrieval (RAPTOR)

Fixes: the answer is the whole document

Flat chunking assumes the answer lives in one passage. For "what's the overall argument of this 80-page report," no single chunk has it. RAPTOR builds a tree: it clusters related chunks, summarizes each cluster, clusters the summaries, summarizes those, all the way up. At query time it retrieves across levels, so it can grab a fine detail or a high-level synthesis.

The paper reported a roughly 20-point absolute accuracy gain on the QuALITY benchmark (long-document QA) paired with GPT-4, state-of-the-art at the time. Treat that as the paper's result on its datasets, not a promise for your corpus.

When to use: long, structured docs where questions span sections, contracts, research papers, filings, technical manuals. Overkill for a FAQ.

5. GraphRAG

Fixes: relational and corpus-wide questions

Vector RAG is blind to relationships. "How are these two people connected," "what themes run across all our incident reports," these aren't similarity questions, and no top-k finds them. Microsoft's GraphRAG uses an LLM to extract entities and relationships into a knowledge graph, then runs community detection (the Leiden algorithm) to cluster the graph and pre-summarize each cluster bottom-up.

Two modes: local search starts from entities near your question and walks the graph (cheap, often one LLM call). Global search answers "what are the themes" by map-reducing over every community summary, powerful and expensive, one trace showed 10+ LLM calls and ~200K tokens for a single global query vs ~11,500 for a local one.

The catch: indexing. Building the graph is hundreds of LLM calls over your corpus up front. Microsoft later shipped LazyGraphRAG to cut that. Use it when relationships and global synthesis are the actual job and your corpus is stable enough to amortize the indexing. Don't put it in front of a help desk.

6. Self-RAG and Corrective RAG

Fixes: retrieved, then wrong anyway

Sometimes retrieval returns confident garbage and the model answers from it. These two add a check.

Self-RAG trains the model to emit "reflection tokens": it decides whether to retrieve, judges whether each passage is relevant, and critiques whether its own answer is actually supported. The judgment lives inside the model.

Corrective RAG (CRAG) puts the judgment outside: a lightweight evaluator scores retrieved docs as correct, ambiguous, or incorrect. On a bad score it doesn't plow ahead, it triggers a fallback (typically web search) to find better sources first. It bolts onto an existing pipeline more easily than Self-RAG, which needs trained tokens.

When to use: high-stakes answers where a confident wrong answer is worse than "I couldn't verify this", support, compliance, medical, legal. Costs extra latency per query, so reserve it for questions that warrant it.

7. Agentic RAG (and Adaptive Routing)

Fixes: no single pipeline serves every query

The six above are tools. Agentic RAG is the workflow that picks and chains them. Instead of one-shot retrieve-then-answer, an agent plans: decides what to look up, retrieves, checks whether that was enough, retrieves again with a better query, maybe calls a second index, and verifies before answering. Multi-step questions become possible.

The danger: running an agent loop on every query is slow and expensive, and most questions don't need it. Which is why the emerging 2026 default is Adaptive RAG: a cheap classifier sits at the front and routes each query to the lightest pipeline that can answer it. Simple lookup → hybrid search. Relational → graph. Multi-step → agent. You pay for complexity only when the question demands it.

The whole map

So which one do you build?

Not all of them. The honest decision rule, in order:

1. Everyone: hybrid search + reranking. Skip this and nothing else matters.
2. Recall still patchy? Add query transformation. One extra call.
3. Chunks losing meaning? Add contextual retrieval. Best cost-to-benefit in the list.
4. Questions span whole documents? RAPTOR.
5. Questions relational or corpus-wide? GraphRAG, if you can afford the indexing.
6. Confident wrong answers hurt? CRAG or Self-RAG.
7. Several query types in one product? Put adaptive routing in front and let it dispatch.

One caution on the numbers here: the contextual-retrieval drop, RAPTOR's QuALITY gain, GraphRAG's token costs, these come from individual papers and vendor posts, not one controlled head-to-head benchmark. There isn't a clean 2026 study running all seven on the same corpus. Treat them as directional, and run the only benchmark that decides anything: take ten real questions from your own users, define what a correct answer is, and test the architectures against that.

The skill that separates a toy demo from a production RAG isn't knowing the fanciest architecture. It's diagnosing which of the five failures is yours, and reaching for the one tool that fixes it. Naive RAG failed you silently. The fix starts with naming how.

Sources: Anthropic, "Introducing Contextual Retrieval" (Sept 2024); Sarthi et al., RAPTOR (ICLR 2024); Microsoft GraphRAG; Asai et al., Self-RAG (arXiv:2310.11511); Yan et al., Corrective RAG (arXiv:2401.15884); Rackauckas, RAG-Fusion (arXiv:2402.03367); Gao et al., HyDE (2022). Figures are each source's own reported results, not a unified benchmark.

Which failure is yours? Drop it in the comments, happy to help you map it to the fix.