"Don't reveal it" is not an access control. RAG permissions belong in retrieval.

The gap between the demo and production

In the tutorial version of RAG, every document in the index is fair game. You embed a folder, you run a similarity search, you stuff the top five chunks into the prompt, and the answer looks great.

In production, the folder is a company. It has HR files, board decks, customer contracts, incident reports and a wiki where half the pages are restricted to one team. The moment more than one person with different access can query the same index, RAG stops being a search problem and becomes an authorization problem.

The most common mistake I see is trying to solve that authorization problem in the prompt:

You may only use documents the user is permitted to see.
Do not reveal confidential information.

By the time the model reads that instruction, the confidential chunk is already in its context window. You are asking a probabilistic text generator to un-know something you just handed it. Sometimes it will comply. Sometimes a well-phrased follow-up, a summarization request or a prompt injection inside another document will get it out. Either way, the access decision has already been made, and it was "yes."

Here are the places this tends to break.

Failure 1: filtering after retrieval instead of during it

A pattern that looks reasonable:

chunks = vector_store.similarity_search(query, k=20)
allowed = [c for c in chunks if user_can_read(user, c.metadata["doc_id"])]
context = allowed[:5]

This is better than nothing, but it has two problems. First, if the filter has a bug or a missing branch, the fallback is "everything." Second, top-k is computed over documents the user can't see, so a restricted folder full of near matches can push every allowed result out of the window and leave the user with an empty or degraded answer.

Push the permission into the query itself so the store never returns what the user can't read:

chunks = vector_store.similarity_search(
    query,
    k=5,
    filter={"acl_groups": {"$in": user.group_ids}},
)

Now the retriever is the enforcement point, and the safe default (no matching groups) is "nothing."

Failure 2: permissions frozen at index time

Illustrative scenario: an engineer moves from the payments team to the mobile team on Monday. The source system revokes their access to the payments design docs the same day. The vector index was built last Thursday, with the old ACL copied into each chunk's metadata, and it re-syncs weekly.

For four days, the RAG assistant still treats that engineer as a payments team member.

Indexing a document also means indexing its access rules, and those rules change on a different schedule than the content. You need a plan for ACL drift: sync permission changes faster than content changes, resolve group membership at query time rather than baking user IDs into chunks, or check the source system for sensitive collections before returning results.

Failure 3: chunks that lose their parent's ACL

A document gets split into 40 chunks. An attachment gets extracted and indexed separately. A nightly job generates summaries of long threads and indexes those too.

Every one of those derived artifacts needs to inherit the most restrictive permission of whatever it came from. A summary of three documents where one is restricted is a restricted summary. If your ingestion pipeline treats derived content as new, permission-free content, you have built a laundering step.

Failure 4: treating citations as proof of authorization

Citations are a good feature. They let users check the answer against the source. But a citation answers "where did this come from?", not "was this person allowed to receive it?"

An answer that cites a document the user can't open is not a broken link. It is a disclosure that has already happened, with a footnote.

How to actually test it

Unit tests on user_can_read() are necessary but not enough. The test that catches real leaks looks like this:

1. Pick a query that touches restricted material.
2. Run it as several roles: admin, team member, contractor, a user whose access was revoked yesterday.
3. For each run, log and compare the retrieved chunk IDs that entered the model's context, not just the final answer text.

The final answer can look identical across roles while the context underneath it is different. A model that "chose not to mention" a restricted chunk still had it, and you should treat that as a failed test.

The reframe

RAG access control is just ordinary authorization, applied at the step where data moves from storage into a prompt. The prompt is where you describe the behaviour you want. Retrieval is where you enforce what's allowed.

How are you handling ACL changes in your RAG indexes today: query-time resolution, fast permission sync, or something else entirely?