Your RAG Answer Can Be Correct—and Still Leak Data

A RAG assistant answers a question accurately and includes the right citation.

There is still a critical question to ask:

Was this user allowed to read that source?

Imagine an internal assistant searching engineering runbooks, customer contracts, and HR documents. All three may contain relevant information. They do not necessarily share the same audience.

Retrieval relevance and access permission are separate decisions.

The failure starts before generation

Consider this pipeline:

User question
    → Search the entire document collection
    → Send matching chunks to the model
    → Ask the model to hide restricted information

The restricted content has already reached the model. A prompt instruction cannot replace an authorization check.

The application should establish which content the user may access before constructing the generation context.

Microsoft's document-level access control documentation describes approaches for restricting search results using document permissions and security filters. The exact implementation depends on the data source and search platform.

Carry permissions into the retrieval layer

When a document is split into chunks, its access rules must remain enforceable.

For example, a chunk might carry metadata such as:

{
  "chunk_id": "runbook-42-chunk-3",
  "document_id": "runbook-42",
  "tenant_id": "tenant-a",
  "allowed_groups": ["platform-operations"],
  "permission_version": 7
}

This is illustrative metadata, not a complete authorization implementation.

The backend must derive the requesting user's identity and permissions from a trusted authentication system. Accepting an arbitrary tenant ID or group list from the browser would undermine the control.

A useful flow is:

Authenticate user
    → Resolve tenant and access permissions
    → Retrieve within the authorized scope
    → Build context from authorized chunks
    → Generate answer
    → Return permitted citations

The authorization rule must apply across every retrieval path, including keyword search, vector search, and fallback queries.

Permission changes are data changes

A document can remain unchanged while its permissions change.

Someone moves teams. A customer contract becomes restricted. A user loses access to a workspace.

If the retrieval index still contains yesterday's permissions, today's answer may expose information the user can no longer read.

Define how permission updates reach the index, how quickly revocations take effect, and what happens when permission information is missing or stale.

The same question applies to cached answers: can a response generated for one user be returned to another? Cache reuse must respect the requesting user's current access rights.

Test the denial paths

A useful evaluation set includes more than questions the assistant should answer.

Check these cases too:

• Two users ask the same question but have different document access.
• Two tenants have documents with similar titles.
• A user's access is revoked after an answer is cached.
• A chunk has missing permission metadata.
• No authorized source supports the requested answer.

Inspect retrieved chunks and citations as well as the final response. A polite refusal does not prove that restricted content stayed out of the pipeline.

Make access boundaries part of answer quality

A production RAG evaluation should ask:

1. Did retrieval find useful evidence?
2. Did the answer accurately represent that evidence?
3. Was every included source authorized for this user?

Getting the first two right is not enough.

How are you testing permission changes and cross-tenant retrieval in your RAG applications?