When a retrieval pipeline copies text out of a document system and into an index, the access list usually stays behind. From then on, every chunk is as visible as every other chunk, and an assistant built on that index can quote a restricted document to anyone who asks the right question. For any team putting an assistant over internal documents, permissions have to be designed into the pipeline from the first ingestion run, not added after a complaint.
How a restricted document reaches the wrong person
Take a chartered accountancy firm with offices in Chennai and Coimbatore that puts an assistant over its shared drive. An articled assistant asks how partner profit shares are calculated. The worksheet that answers it sits in a folder only the managing partners can open. The assistant quotes it, accurately and with a citation.
Nothing malfunctioned. The model receives no information about who is asking. It writes from whatever retrieval hands it, and retrieval searched the whole index because nothing told it not to.
An instruction in the prompt such as "do not reveal confidential information" does not fix this. The model has no way to know what is confidential to this particular person, and a prompt instruction is followed most of the time rather than every time. Access control has to happen before any chunk reaches the model, inside retrieval (the search step, not the generation step).
Carry the access list onto every chunk
The fix starts at ingestion. For every chunk, record who may read it, inherited from the source document. That can be group identifiers, a classification level, a client or tenant identifier, or all three. Store these as metadata next to the vector, in the same row.
- Source systemthe document's owners and groups
- Ingestioncopies the access list onto each chunk
- Indexvector, text, metadata, allowed groups
- Retrievalfilters by the asker's groups
- Modelonly sees permitted chunks
If any layer drops the access list, every layer below it inherits the leak.
At question time, resolve the asker to their groups from the same identity system the document store uses, and pass those groups into the retrieval query as a filter. Most vector stores accept a metadata filter alongside the similarity search. Where the index lives in Postgres, the filter is an ordinary WHERE clause, which we prefer because the rule is then a query that anyone on the team can read and test:
SELECT chunk_id, doc_id, body
FROM chunks
WHERE tenant_id = $1
AND allowed_groups && $2::text[] -- asker's groups
AND is_current
ORDER BY embedding <=> $3 -- question vector
LIMIT 8;
The similarity ranking only ever runs over chunks this person may see.
Filter inside the query, not after it
The tempting shortcut is to retrieve the closest chunks first and drop the ones the asker may not see afterwards. It fails in three ways.
- Top results may all be restricted
- Restricted chunks loaded and logged
- Easy to miss on one code path
- Ranking runs over permitted chunks only
- Restricted text never leaves the index
- One place to enforce and test
The first failure is quiet. The articled assistant's question returns eight chunks from the restricted worksheet, the post-filter removes all eight, and the assistant says it has no information, even though a permitted summary of the firm's policy sat at rank nine. The second is the leak. The restricted chunks were fetched, held in memory, written to the request log, and in some implementations sent to the model before the filter ran. The third shows up months later: the filter lives in one function, someone adds a "related questions" feature that calls retrieval directly, and nobody notices.
Treat the filter as a hard condition in code, applied on every retrieval path, including search previews, suggested questions and the citation list.
Keeping permissions in sync
Permissions change. A folder is reclassified, a senior moves from audit to tax, a client engagement ends and its files are locked. The index has to follow, either by re-ingesting on permission change events from the source system or by re-syncing access lists on a schedule. For most internal document sets a nightly sync is acceptable to the business. For payroll, client financials or anything regulated, ask the document owner what delay they will accept, and write the answer down.
Watch the account that runs ingestion. It needs to read everything in order to index it. The query path must never run as that account; it runs as the person asking, with their groups. Mixing the two is how a service account's read-everything permission quietly becomes every user's permission.
Shared indexes and tenants
When one index serves several customers, such as a product sold to forty coaching institutes, the tenant identifier is the first filter on every query, enforced in code and never left to the prompt. Where the stakes are higher, use a separate index or namespace per tenant. Then a missing filter returns nothing, instead of returning another institute's student records.
Where access control leaks
The common mistakes are mostly about what was left out. The index was built by a service account that reads everything, and queries run with no user context at all. Permissions were copied once at ingestion and never re-synced, so a person who left the tax team in March can still ask about its client files in September. Filtering is by document when sharing is by section, so a page with a public half and a restricted half leaks the restricted half.
Three more are less obvious. The answer is filtered, but the sources list still shows titles of documents the asker may not open, and a title can say a great deal on its own. A cached answer is served to a different user with different permissions, because the cache key was the question text alone. And testing was done with an admin account, so everything worked and nothing was ever restricted. The fix for that last one belongs in the evaluation set, as described in measuring a RAG system: include questions that a narrow user should be refused.
The same thinking applies once a model can act as well as answer. Permissions for agents covers what an agent may read, write and ask about.
Checks before go-live
- Confirm every chunk carries the identity of who may read it.
- Confirm the retrieval query includes the asker's groups as a filter, on every code path.
- Agree with each document owner how quickly a permission change must reach the index.
- Make the answer, the citation list and the cache key all depend on the asker.
- Log which chunks were sent to the model for each answer.
- Run the evaluation set as a user with narrow permissions and review what they see.
Do the first two before anything else: tag the chunks and put the asker's groups in the query. Then test with the narrowest real account you have, a new joiner in a branch office, rather than the account of whoever built the system. For how retrieval fits into the rest of an assistant, see what a RAG pipeline does.
