Permission-aware RAG: enforce access at retrieval, not in the prompt

Most internal knowledge assistants start the same way: index everything in the shared drive, put a chat box on top, and add "only answer using documents the user is allowed to see" to the system prompt.

That last line is not access control. It's a suggestion to a language model. If a restricted document is retrieved and placed in the context, the model has already seen it, and no instruction reliably stops it from leaking a number, a name or a summary.

The rule that fixes this is simple: a user must never be able to retrieve a chunk they couldn't open in the source system. Here is how to build that.

1. Carry permissions into the index

When you ingest a document, store its access-control list with every chunk: the user IDs, group IDs or roles that can read it in the source system (SharePoint, Google Drive, Confluence, your CRM). Store them as filterable metadata, not as text in the chunk.

{
  "chunk_id": "policy-2291#4",
  "text": "...",
  "source": "sharepoint://hr/policies/leave.docx",
  "allowed_groups": ["hr-all", "managers-emea"],
  "last_modified": "2026-10-30T09:12:00Z"
}

2. Filter inside the search, not after it

At query time, resolve the user's identity (from your SSO token) into the groups they belong to, and pass that as a filter in the vector or hybrid search itself. Most vector databases support metadata pre-filtering.

Avoid post-filtering, where you retrieve the top 10 and then drop the ones the user can't see. It leaks through ranking (the user may get fewer, worse results that hint at what exists), and it breaks easily when someone changes k.

3. Sync permission changes as fast as content changes

This is the part teams forget. Someone is removed from a group, or a folder is locked down, and the index still holds the old ACL until the next full re-sync, sometimes days later.

Treat ACL changes as first-class events: subscribe to the source system's change notifications where they exist, re-sync permissions on a short schedule where they don't, and alert when an ACL sync job fails. A stale permission is a data leak with a delay.

4. Don't let the answer step widen access

Retrieval is not the only path. Watch for:

• Answer caches keyed only by the question text. Two users with different permissions ask the same question and get the same cached answer. Key the cache by question plus permission set, or don't cache restricted content.
• Conversation memory that carries a restricted chunk into a later turn after the user's access changed mid-session.
• Tools and agents that call APIs with a service account. If the agent can query the CRM with admin rights, your retrieval filter doesn't matter. Tools should act with the end user's delegated permissions.

5. Make citations part of the contract

Require every claim in the answer to cite a chunk ID that was actually retrieved for this user. Validate the citations in code before showing the answer: if a citation points to a chunk outside the retrieved set, drop the claim or refuse.

This gives you two things at once: users can verify answers, and you have an audit trail of exactly which documents influenced each response.

6. Test it like a security feature

Add permission tests to your evaluation set, not just quality tests:

• A user in group A asks about a document only group B can see. Expected: "I couldn't find that in the documents you have access to." Not a summary, not a hint that it exists.
• Remove a user from a group, wait for one sync interval, re-ask. Expected: access gone.
• The same question from two users with different access returns different, correct answers.

Run these on every change to the ingestion pipeline, the chunker or the cache.

A short checklist

• [ ] ACLs stored as metadata on every chunk
• [ ] Identity resolved from SSO at query time
• [ ] Permission filter applied inside the search query
• [ ] ACL changes synced on a short, monitored schedule
• [ ] Caches keyed by permission set
• [ ] Tools act with delegated user permissions
• [ ] Citations validated in code
• [ ] Permission test cases in the eval suite

None of this is exotic. It's the same rule we already apply to databases and file shares, applied to a new kind of search. The prompt can shape how the model answers; it should never be the thing deciding who can see what.

I'm VP Marketing at SoluLab, where our teams build RAG systems and AI agents for enterprises. This post reflects the patterns we design for; your stack and compliance needs will shape the details.