A RAG Index Is a Materialized View, So Stop Treating It Like a Cache

TL;DR — Enterprise RAG indexes are derived data over lakehouse tables, which makes them materialized views with all the maintenance problems that implies. Cron re-embedding, a single freshness number, and text copied into the vector store all fail in predictable ways. Store pointers and content hashes, hydrate from the source at query time, and pin answers to a snapshot.

Ask a team how their enterprise RAG index stays fresh and you will usually hear the word "refresh." A nightly job scans the lakehouse, chunks documents, embeds them, and upserts into a vector store. When someone complains the assistant quoted last quarter's numbers, the fix is to run the job more often.

That is cache thinking, and it is the wrong mental model. A cache is a disposable copy of something you can always re-fetch. A RAG index over lakehouse data is a derived dataset with its own schema, its own cost to rebuild, and its own correctness contract. That is a materialized view. Once you say it that way, most of the hard problems already have names.

The unit mismatch nobody budgets for

The lakehouse tracks change at the level of files, partitions, and rows. Retrieval operates on chunks. Embedding is the expensive step, so you want to re-embed only what actually changed. But a changed row does not tell you which chunks changed, and a rewritten file does not tell you which rows did.

Naive pipelines resolve this by re-embedding everything touched by a changed file. Compaction and rewrite operations, which are routine table maintenance, then look like mass edits. Your embedding bill spikes because someone optimized file layout, not because any content changed.

The fix is the same one incremental view maintenance has always used: a stable key and a content hash.

• Every chunk gets a deterministic ID derived from the source record key and its position or span within it.

• Every chunk stores a hash of its normalized content plus the chunking and embedding configuration.

• On each run, compute the desired set of (ID, hash) pairs from the change feed, diff against the index, and embed only the delta.

A file rewrite that changes no content produces zero embedding calls. A chunking-config change produces a full rebuild, and it should, because that is a schema migration, not a refresh.

Freshness is a vector, not a number

"The index is 20 minutes behind" is almost never a true statement. An index fed by five source tables has five watermarks, and they advance independently. The CRM table may be seconds old while the contracts table last landed yesterday because an upstream job failed.

This matters because RAG answers routinely join facts across sources. If one retrieved chunk reflects Tuesday's pricing and another reflects Thursday's discount policy, the model will confidently synthesize a statement that was never true at any point in time. This is a torn read, and databases solved it decades ago with snapshot isolation.

Table formats with snapshots give you the primitive. Practical consequences:

• Track a watermark per source, expressed as the snapshot or version of the table that was last fully indexed.

• Expose it at query time. Retrieval results should carry the source version each chunk came from, so the application can see that the evidence spans different points in time.

• Surface staleness to the model and the user. An "as of" annotation in the context costs a few tokens and prevents a class of silent errors. For some queries, such as anything involving balances or inventory, a stale source should trigger a fallback to a live query rather than a hedged answer.

Not every question needs consistency across sources. But you cannot decide which ones do if the index does not tell you what it knows and when it knew it.

Store pointers, hydrate at read time

The most consequential design choice is what the vector store actually holds. The default is to put the chunk text next to the embedding and return it directly. It is simple and fast, and it makes the index a second copy of your data that has to be kept correct forever.

The alternative is late binding. The index stores the embedding, the chunk ID, the source key, and the content hash. Vector search is used only for candidate generation. Before anything reaches the prompt, a hydration step fetches the current record from the lakehouse by key and checks the hash.

That one step buys three things at once:

• Freshness repair. If the hash no longer matches, the chunk is stale. Drop it, or re-chunk that record inline, and enqueue it for reindexing. The index can lag without the answer lagging.

• Deletion correctness. A record deleted at the source simply fails to hydrate. You are no longer racing a tombstone through the pipeline to stop leaking something that should be gone.

• A single enforcement point for access. Hydration goes through the same governed read path as every other consumer, with row filters and column masks applied using the caller's identity. The index never has to model your permission system, which it cannot do faithfully anyway.

The cost is real. Hydration adds a read per candidate, and you will over-fetch candidates because some will be filtered out. Over-fetch factors and tail latency of the hydration store become things you tune. I think that is a good trade. Latency is a budget you can spend; a leaked or fabricated fact is not recoverable by tuning.

What breaks when you ignore this

The failures are quiet, which is why teams tolerate them for months.

• Phantom facts. Updated records leave old chunks behind because the upsert keyed on position, not stable ID. Retrieval returns both versions and the model picks one.

• Embedding cost drift. Maintenance on the lakehouse turns into re-embedding on the index, and nobody connects the two.

• Unreproducible answers. Someone asks why the assistant said X last week. Without snapshot versions on the retrieved evidence, you cannot reconstruct what the index returned, so you cannot debug it or audit it.

• Silent partial outages. One upstream table stalls. The index is "up," retrieval works, and one slice of the corpus is simply getting older. Without per-source watermarks, no alert ever fires.

The design checklist

If you are building or auditing a lakehouse-backed RAG system, ask these questions in order:

• Can I derive every chunk deterministically from (source key, source version, chunk config, embedding model)?

• Does the index update from a change feed, with content hashes preventing no-op re-embedding?

• Do I have a watermark per source, and does retrieval output carry it?

• Does the final prompt contain only text that was hydrated from the source under the caller's identity?

• Can I replay a past query against a past snapshot?

None of this is novel. It is view maintenance, snapshot reads, and late binding, applied to a new kind of derived table. The novelty is that the consumer is a language model that will happily paper over every inconsistency with fluent prose. Databases surface staleness as wrong numbers; LLMs hide it. That is the strongest argument for building the discipline into the index itself, because the model will never tell you the index is out of date.