How do you handle multi-tenant data isolation in RAG?

Multi-tenant data isolation in RAG ensures that each organization (tenant) can retrieve and generate answers using only the documents it is authorized to access. The isolation rule must apply throughout ingestion, indexes, retrieval, caches, logs, and answer generation—not just in the user interface.

Beginner-friendly example

A SaaS support assistant serves Company A and Company B. Both upload a document titled “Pricing Policy.”

• An employee of A asks: “What is our discount limit?”
• Retrieval must consider only A's authorized policy, even if B's document has a higher vector similarity score.
• The answer may cite A's policy but must never expose B's prices, passages, or metadata.

Common architecture options

1. Separate indexes or collections per tenant: Stronger operational separation and simpler tenant-scoped queries, but many small tenants may create expensive index-management overhead.
2. Shared index with mandatory tenant filters: Store a trusted tenant_id on every chunk and apply it in the search engine *before* ranking or returning results. Efficient for many tenants, provided enforcement is reliable.
3. Hybrid isolation: Use dedicated indexes for larger or regulated tenants and a carefully partitioned, filtered index for smaller ones.

The right design depends on tenant count, scale, contractual rules, costs, and the vector engine's filtering guarantees.

How to implement safely

1. Authenticate the user: Derive tenant identity from validated credentials/session claims, not from a tenant ID supplied inside a chat message.
2. Authorize the request: Establish document-level permissions, user roles, and any cross-team sharing rules.
3. Tag data at ingestion: Attach tenant ID, document ID, permission scope, source version, and deletion state to every stored chunk and embedding.
4. Enforce retrieval restrictions: Filter authorized tenants and documents in the database or search service before evidence reaches the model. Avoid searching globally and trusting the LLM to ignore unauthorized matches.
5. Protect derived artifacts: Tenant-scope retrieval caches, embeddings tied to private documents, reranker candidates, conversation history, and any generated summaries.
6. Verify downstream context: Ensure every passage provided to the LLM passes the same authorization policy, including during graph traversal and multi-source search.
7. Audit and test: Log safe tenant-scoped identifiers and denials. Test missing filters, forged IDs, cross-tenant near-duplicate documents, cache collisions, and permission changes.

Small numerical example

Suppose a shared vector collection has 1 million chunks, with 400,000 for A and 600,000 for B.

An A-user's search for Top-K = 5 must return up to 5 authorized results from A's eligible 400,000, not the global top five with B's results removed afterward. The latter can reduce recall and, if filtering is misplaced, risk unauthorized exposure.

Common mistakes

• Trusting an LLM prompt such as “only use Company A documents” as a security boundary.
• Using an untrusted tenant_id parameter without validating ownership.
• Filtering only after retrieval or generation.
• Reusing cached contexts across tenants.
• Updating permissions in the primary database but leaving stale vector-index access rules.
• Assuming tenant isolation alone handles finer user- or document-level authorization.

Interview takeaway: Multi-tenant RAG must enforce a trusted tenant boundary at every retrieval and data-processing layer. Shared indexes can work, but tenant filters and permission checks must be server-enforced, testable, and fail closed.