A Field Guide to RAG Failure Modes: Garbled Scans, Duplicate Chunks, and Leaky Indexes

You have been here before. The prototype demoed well. You shipped the internal Q&A tool, and within a month the chat channel about it reads like a support queue: "it answered with a paragraph from an unrelated contract," "it gave me three copies of the same page," "wait, why does this return customer phone numbers?"

The instinct is to debug the model layer: swap embedding models, retune chunk size, bolt on a reranker. Sometimes that helps. More often the root cause is upstream of the embeddings entirely, in the document pipeline. This piece is a field guide to three failure modes I run into often, why each one is structurally invisible in demos, and what the fix does to storage and token cost.

Failure mode 1: scan rot

A large share of enterprise documents are images wearing a PDF costume. Scanned contracts, photocopied HR files, faxes that were digitized by photographing a printout. Run a standard chunker over them and it happily slices the pages, computes embeddings over whatever text the OCR layer emitted, and stores the result. Here is what a real chunk from a degraded scan can look like:

...pursuant to the Agreement, the Vendor shall pay
1?.3?4?5?6?.78 within 30 (3O) days of invoice...
[SEAL overlaps signature block]
[unreadable line — table column lost]

The failure is silent by construction. The embedding of noisy text still lands close to the right region of vector space, so retrieval returns the chunk with a perfectly respectable cosine similarity. The generator then produces a confident answer over a corrupted premise. No exception is thrown anywhere in the stack. Retrieval quality just drifts, in ways that no error budget will ever catch.

Three sub-species worth knowing:

• Tables rendered as pixels. A pricing table that exists only as an image becomes prose soup after OCR; the row/column relationships that made it meaningful are gone.
• Seals, stamps, and stamps over fields. In some markets — Chinese government and finance documents, for example, where the OFD layout format is standard — official seals are placed directly across the fields you care about. Generic OCR pipelines choke on this.
• Confidence-score theater. OCR engines report confidence, and teams treat a high average confidence as proof the text is fine. Averages hide exactly the fields that matter. Read the output by eye, sample pages at full zoom, before anything enters the index.

The fix is structure recovery before chunking: rebuild the document's logical layout (headings, tables, reading order) so chunks are composed of sentences rather than smears. This is a data-engineering step, not a model step, and it is where a large fraction of "the RAG answers are weird" tickets actually originate.

Failure mode 2: duplicate chunks

Vector stores do not deduplicate. Consider the life cycle of an important contract: draft, reviewed draft, final, final-final, plus the copy someone emailed to a vendor in March. All five versions describe near-identical content, so their embeddings cluster tightly. When a query lands in that region, the top-k window fills with clones of the same passage — and the unique, authoritative passage that actually answers the question gets crowded out by its own siblings.

This is a diversity and ranking problem, and it compounds a statistical fact about corporate corpora. A survey commissioned by M-Files found that 96% of employees struggle to find the most current version of a document, and 83% have had to recreate one because they couldn't find the original. In other words: the corpus you are indexing was produced by a process that manufactures duplicates. They are not an edge case; they are the native state.

Practical countermeasures, in increasing order of effort:

1. Exact-duplicate removal via content hashing — cheap, do it first.
2. Near-duplicate clustering via embedding similarity, keeping one canonical chunk per cluster.
3. A canonicalization policy: decide which version of each document is the truth, and demote the rest with metadata rather than deletion.
4. Source-metadata weighting at query time, so retrieval prefers the authoritative copy.

None of these require touching the model. All of them change what the top-k window looks like, which is the only thing the answer generator ever sees.

Failure mode 3: leaky indexes

Retrieval is an access-control bypass waiting to happen. Document-level permissions do not survive chunk-level indexing: the vector store flattens "only the finance team can open this file" into "anyone whose query matches this embedding gets the passage."

Worse, the sensitive fields do not need a sophisticated attack to escape. A question about a named account retrieves the chunk that contains the account's phone number, because it matches. The leak is the library itself.

The industry data puts hard numbers on the surrounding risk. IBM's 2025 Cost of a Data Breach report found that one in five companies has suffered a breach tied to employees using AI tools on their own — the pattern called shadow AI — and that these organizations paid an average of $670,000 more per breach than their peers. Among breached organizations, 63% had no formal AI governance policy. IBM's same report also found customer personally identifiable information made up 53% of the data stolen in breaches — precisely the category that lives inside contracts, invoices, and HR files.

Two design rules follow. First, detect and mask sensitive fields before embedding and indexing, not in a post-hoc filter that is always one query behind. Second, masking has to run inside your own environment — shipping raw documents to a third-party masking API just recreates the leak one hop earlier in the pipeline. Rule-based detection over common categories (ID numbers, bank cards, phone numbers) plus industry-specific structures is the workable baseline, and the rule set should be tunable, because every document mix is different.

The three-stage fix

Put together, the upstream pipeline is short to describe: clean → mask → compress → index.

• Cleaning: exact-duplicate hashing, near-duplicate clustering, layout and table recovery for scans, normalization of encodings and whitespace.
• Masking: deterministic detection of sensitive fields, masking applied before embedding, executed locally.
• Compression: fidelity-preserving reduction of the files that dominate storage, keeping text and layout intact.

One implementation of this stage ships as a "RAG data engineering base" from UglyPear Data — search for the name if you want a reference architecture — but the sequence applies regardless of vendor, and a strong in-house team can build all three stages from primitives.

What the fix does to the numbers

Storage first. IDC attributes 80% to 90% of enterprise data to unstructured formats, and Komprise's surveys report that 60% to 70% of the data on enterprise NAS hasn't been accessed in over 90 days while sitting on the most expensive tier. Scan-heavy documents are the heavy tail of this: a few-hundred-page scanned contract can weigh hundreds of megabytes raw.

Fidelity-preserving compression changes the economics directly. Compression engines that strip redundant data without touching text cut file size by 60% to 80% across PDFs, images, Office documents, and scans. Run the arithmetic on a 2 TB scan-heavy library: that is roughly 400 GB to 800 GB after preprocessing, before you count the smaller vector store and the reduced embedding batch sizes.

Token cost is the second, less obvious win. Multimodal tokenizers bill heavily for image content, and retrieval pipelines that pass figures and scanned pages to the model inherit that bill on every call. Measured on a GPT-4o basis, image-heavy content cut token costs by 85.9% after compression — and unlike a one-time storage saving, per-token savings compound across every query, every day, for as long as the library runs.

Evaluating any preprocessing pipeline: five dimensions

Whichever pipeline you adopt — built or bought — judge it on these axes rather than on feature lists:

1. Fidelity: does the text survive untouched, and does layout (tables, seals, reading order) come through?
2. Masking coverage and locality: which sensitive categories are detected, and does the processing run inside your network?
3. Throughput: pages per second at your actual volume, including pathological formats.
4. Integration: does the output plug into the vector store you already run, without a rebuild?
5. Observability: can you sample before/after pairs at full zoom and diff the extracted text?

A pre-flight checklist

• Sample OCR output by eye, at full zoom — never trust average confidence.
• Measure the duplicate ratio of retrieved chunks in a test query set.
• Grep your index for PII patterns; assume you will find some.
• Compute storage and token cost per query before and after compression.
• Keep masking inside your own network.

The model gets the attention, but the pipeline gets the answers. Debug upstream.