RAG access control: permissions that survive retrieval
Short answer: RAG access control has to happen inside the retrieval query, not after it. If the retriever returns a passage the user is not entitled to see and the model is instructed not to mention it, the passage is already in the context window and you are relying on model behaviour for a security boundary. Permissions belong on the chunk, applied as a filter at query time, and the decision has to be made before the index is built.
Why post-filtering is not access control
Most RAG access control designs start in the same tempting place: retrieve the top passages, check which ones the user may see, drop the rest, generate an answer. It is easy to build and it is wrong.
Two things go wrong, and the second is worse.
If you drop passages after retrieval, your top-k has been consumed by documents the user cannot see, so the answer is assembled from whatever survived. A user with narrow permissions gets a worse answer than they should, for reasons nobody can explain from the logs.
And if you pass the passages to the model with an instruction to ignore the unauthorised ones, you have not implemented access control at all. You have written a request. The content is in the context window, it influences generation whether or not it is quoted, and extraction under adversarial prompting is a well-documented behaviour rather than a theoretical risk.
The boundary has to be the query. RAG access control means the retriever is incapable of returning what the user may not see, rather than well-behaved about it afterwards.
Permissions belong on the chunk
This is the part that surprises teams, and it is why retrofitting RAG access control onto a live index is usually a rebuild.
A document has permissions. A chunk of that document inherits them, and the inheritance has to be recorded at ingestion, because at query time you have a vector and a filter and no memory of where the text came from unless you stored it.
So every chunk carries the access metadata of its source: the groups, roles or principals entitled to it, in whatever model your source systems use. Then retrieval is a filtered vector search, with the filter derived from the authenticated user’s own entitlements rather than from anything the conversation asserted.
Most vector databases support metadata filtering. What varies enormously is whether the filter is applied before or during the search rather than after it, and the difference shows up as both a correctness problem and a latency problem at scale. Ask the question explicitly when you choose a store; the documentation is often vague and the behaviour is not.
The hard part is that permissions change
A static access model makes RAG access control straightforward. Real ones are not static.
Someone leaves a team. A document is reclassified. A project folder’s inherited permissions change at the parent. A contractor’s access expires. Each of these has to reach your index, and the index is a copy of data whose authority lives somewhere else.
Three approaches, in ascending order of cost and correctness.
Re-index on a schedule. Simple, and it means your permission state is stale by up to one cycle. Acceptable for slow-moving internal documentation, not acceptable for anything where a revocation has to take effect promptly.
Event-driven updates. Subscribe to permission changes in the source system and update chunk metadata as they arrive. Correct, and it requires the source system to emit those events reliably, which not all of them do.
Resolve at query time. The strictest form of RAG access control: store a stable identifier on the chunk and check entitlement against the live source during retrieval. Always correct, and it costs you a round trip per query, or a cache that reintroduces the staleness you were avoiding.
Most production systems end up with a hybrid: event-driven where the source supports it, scheduled reconciliation as a backstop, and query-time resolution for the small set of documents where a stale permission would be a serious problem. Deciding which documents are in that set is a business conversation rather than a technical one.
Multi-tenancy is a special case worth getting right early
If your system serves several customers from one deployment, tenant isolation is a stricter requirement than user permissions within a tenant. A cross-tenant leak is an incident with contractual consequences; a within-tenant over-share is a bug.
Treat them differently in the architecture. Tenant separation should be structural — separate indexes, separate namespaces, or a partition key enforced below the application layer — rather than another metadata filter that a mistaken query could omit. Within a tenant, filtering on user entitlements is the right mechanism.
The test to run before launch: construct a query as tenant A designed to surface tenant B’s content, and confirm the failure is structural rather than a filter that happened to be present.
What the audit trail has to record
When someone asks what the assistant knew, the transcript is not enough.
Record, per query: who asked, the filter that was applied, which chunks were retrieved with their source document identifiers, which of those made it into the context window, and what was generated. That set reconstructs the decision. Any subset of it leaves a question you cannot answer.
This matters beyond security. Audit trails are the mechanism by which you debug a wrong answer months later, and the reason you can tell a customer whether their data was ever used to answer someone else’s question. Systems built without them cannot answer either question, and the absence is only discovered when the question is asked.
The same discipline runs through enterprise RAG architecture generally, and it is the reason permissions are a week-one design item in our custom RAG development services rather than a hardening phase.
Proprietary data raises a separate question
Access control governs who may see what. There is a second question, frequently conflated with it, about where the data may go.
If your retrieval corpus contains proprietary or regulated content and your generation step calls a hosted model, that content leaves your network on every query. Whether that is acceptable depends on your agreements, your regulator and your risk appetite, and it is a decision to make deliberately rather than inherit from a default.
The options are the usual three: a hosted provider that will sign an appropriate agreement and commits to not training on your inputs, a model deployed inside your own network, or de-identification before the content reaches the model. Each has a real cost. The write-up in on-premise RAG covers the second in detail.
A build order that works
Settle RAG access control before the first chunk is indexed. Not the first prototype, which can use public content, but the first index that will ever hold restricted data.
Then, in order: decide how chunk permissions are represented and where they come from, confirm your vector store filters during search rather than after, build the entitlement resolution path from your identity provider, wire the audit trail, and only then tune retrieval quality.
Teams that invert this get a system with excellent answers that cannot be deployed, and the fix is not a patch.
The takeaway
RAG access control is an ingestion problem that presents as a query problem. Permissions ride on the chunk, the filter runs inside the search, the model is never handed content the user could not open themselves, and the audit trail records what was retrieved rather than only what was said. Every one of those is cheap to design in and expensive to add.
EpochC builds custom RAG development services with permission-aware retrieval and audit trails from the first commit. See the graph and vector retrieval case study, or start a project.