RAG Metadata Filtering and Access Control: A Safe Design

Map each policy to an enforceable field

Enforce policy before relevance ranking

Authorization begins outside the search engine: authenticate the caller, derive tenant and group claims from a trusted identity boundary, evaluate product policy, and translate the decision into a bounded filter. Never accept a tenant ID or clearance label merely because the user included it in the question.

Azure AI Search's official filter guidance explains that filters operate on nonvector fields, can qualify documents used by keyword or vector retrieval, and are used for security filters. It also notes that filters use exact value criteria. Normalize identifiers during ingestion and generate filter expressions through a typed policy layer rather than free-form model output.

Filter timing has security and relevance consequences

Search platforms may offer pre-filter and post-filter modes for vector queries. For access control, the externally observable result must always contain only authorized evidence. A mode that considers forbidden candidates before removing them also needs review for timing, counts, telemetry, and side-channel behavior. Apply one policy consistently across lexical, vector, hybrid, reranking, and fallback paths.

The generator cannot repair an authorization breach

Once restricted content reaches the model context, it may influence the response even if a later validator removes an obvious quotation. It may also appear in traces, caches, citation objects, or debugging payloads. Generation guardrails are useful for response policy, but they are not a substitute for retrieval eligibility.

Record document identifiers and policy decisions without copying sensitive passages into routine metric labels. Validate that citations refer only to context actually supplied to the model and still authorized at response time. If authorization changes during a long workflow, define whether the operation rechecks policy before final disclosure.

Treat freshness and deletion as policy fields

A security-correct answer can still be operationally wrong when it uses a superseded policy. Carry version and effective-time metadata to every chunk, define which revision is current, and test historical questions separately from current-state questions. Deletion must remove source content, child chunks, embeddings, keyword entries, derived summaries, and cached responses within the declared lifecycle.

OWASP's vector and embedding weaknesses guidance recommends fine-grained access controls, permission-aware stores, and strict logical partitioning. It also describes cross-group leakage when shared vector storage is not properly isolated. Use that threat as a negative fixture, not merely a design note.

Failure cases that must deny or alert

• Caller-controlled tenant filter: changing a request field reveals another tenant. Bind scope to authenticated server-side identity.
• Parent-only policy: child chunks omit group labels and become globally searchable. Verify inherited fields during indexing.
• Hybrid asymmetry: lexical search filters correctly while vector search does not. Run the same denial fixtures through every channel.
• Cache cross-talk: an authorized answer is reused for a caller with different rights. Include policy scope and revision in cache identity.
• Stale membership: revoked group access remains in a long-lived session. Define claim freshness and reauthorization points.
• Soft deletion: the source disappears from the UI but remains retrievable through old chunks or summaries. Test every derived artifact.

A safe implementation sequence

1. Write the authorization contract. Define principals, tenants, groups, classifications, purposes, revisions, denial behavior, and recheck points.
2. Design immutable identity fields. Attach tenant, source, policy, version, and lifecycle identifiers to parents and every derived chunk.
3. Build a trusted filter compiler. Convert authenticated claims and policy decisions into typed, bounded search expressions outside the model.
4. Enforce one boundary everywhere. Cover lexical, vector, hybrid, reranking, context expansion, citation lookup, cache, export, and fallback.
5. Create negative fixtures. Probe cross-tenant IDs, revoked groups, missing labels, stale revisions, deleted documents, malformed filters, and empty results.
6. Read back the evidence path. Confirm no denied identifier appears in candidates, context, response, citations, traces, or cache entries.
7. Monitor policy drift. Version identity mappings and filters, audit decisions, and replay denial cases before index or policy releases.

Decision summary

Enforce tenant, role, lifecycle, and freshness policy before relevance ranking and before cached context can cross boundaries. Retrieval authorization is a data-access control; the generator cannot safely redact evidence it should never have received.