The correct document exists in the index but never appears among the retrieved candidates. What debugging sequence would you follow?

Production incident (illustrative): An enterprise RAG assistant is asked, “What is the reimbursement ceiling for contractors traveling to Singapore?” The authoritative 2026 Travel Policy, paragraph 4.3, explicitly answers the question and is confirmed present in the vector collection. Yet the assistant's first-stage top-10 candidates contain only older travel FAQs and unrelated expense policies. The reranker and LLM cannot recover a document they never receive. The team initially increases top-k without investigating *where* the target disappeared.

Key principle: “The document exists in the index” is not the same as “its correct version/chunk is eligible, vectorized in the searched space, visible on the serving replica, reachable by the candidate generator, and passed to downstream fusion.” Debug by finding the first stage that excludes the known-good chunk, not by changing every parameter simultaneously.

1. Freeze a reproducible missing-candidate case

• Record the raw question, tenant_id, authenticated principal, policy effective date, expected document_id/version/chunk_id, index alias, named vector, query embedding model/revision, search distance metric, all filters, top_k, ANN candidate budget, fusion and reranking settings, and precise request trace.
• Validate the expected answer-bearing chunk, not merely the parent document. The correct document may be indexed while paragraph 4.3 is missing, truncated, merged with irrelevant context or represented by an old version.
• Verify this user is *authorized* to retrieve that exact chunk under tenant, ACL, version, and effective-date rules. If it is genuinely disallowed, absence is correct; never remove security filters to make the benchmark green.
• Use a fixed point-in-time snapshot, query vector and query class so every experiment is comparable. Preserve raw candidate IDs and rank scores after each pipeline stage. Diagnostic experiments with filters removed must run only in an isolated privileged test on synthetic or otherwise permitted data, never in production.
• Separate first-stage candidate recall from downstream reranking and answer quality: if the correct chunk never enters the first-stage pool, tuning prompts or reranker models cannot solve the root problem.

2. Trace the first failure boundary in order

• A. Authoritative source and lineage: Does the right policy revision still exist? Trace source record → parsed text → chunk manifest → stable chunk ID → embedding hash/model → vector write receipt. Check OCR, extraction, heading loss, table parsing, chunk boundaries, supersession, tombstones, and partial batches.
• B. Exact point retrieval: Fetch expected chunk_id directly from the *production-target collection* and check payload, vector presence, vector name, model version, tenant, source revision and replica. A successful ID lookup proves stored existence, not ANN search inclusion. Check old aliases, wrong region/shard and stale replicas if lookup disagrees.
• C. Eligibility predicate: Evaluate every mandatory filter on that exact point: tenant, ACL, active_version, date bounds, locale and document type. Compare field types (string versus integer), null/missing metadata, nested-filter semantics and logical AND/OR composition. Maintain authorization gates; repair inconsistent metadata rather than weakening access.
• D. Exact similarity baseline: Within the same eligible corpus and distance metric, run exact/flat kNN where supported or brute-force score a controlled filtered subset. Record the missing chunk's *exact rank* and score. If rank is 213, it is not an ANN failure when top_k=10; the embedding/query representation simply does not place it close enough.
• E. Approximate ANN: If exact rank is 3 but HNSW returns no target in a candidate pool of 50, investigate ef_search/hnsw_ef, Elasticsearch num_candidates, quantization/rescoring, graph health, index readiness, shard health and request timeout. Qdrant supports a diagnostic exact=true; Elasticsearch distinguishes exact and approximate kNN.
• F. Filtering interaction: Inspect whether the same authorized filter is applied during candidate traversal or after top-k selection. Elasticsearch distinguishes filtered approximate kNN from post-filtering, which can yield fewer than k. Qdrant's search planner can choose different strategies per segment and offers filterable HNSW / ACORN options where supported. Preserve filter *semantics*, but consider changing when/how eligibility is enforced within the retrieval engine.
• G. Multi-shard aggregation: If target appears in a local shard's candidate list but not the merged list, examine shard fan-out, per-shard candidate budgets, global size/top_k, partial responses, request deadlines, replicas and routing.
• H. Hybrid fusion and deduplication: If present in dense or BM25 candidates but absent after reciprocal rank fusion (RRF), weighted blending, metadata deduplication, parent grouping, MMR or result thresholds, log before-and-after ranks and IDs. Ensure candidate limits are sufficient before fusion; verify document_id grouping does not accidentally collapse the answer-bearing version.
• I. Reranker/context assembly: Only if the candidate survives first-stage retrieval, investigate reranker truncation, cross-encoder ordering and limited context budgets. That is a later failure stage, not evidence that initial retrieval failed.

3. Root-cause decision tree

• Point ID missing or wrong vector name: index/lineage/namespace failure; replay only the missing active version and verify visibility.
• Point present but filter-ineligible: validate business policy; if the record is permitted, repair payload/ACL/version metadata and filter mapping. Keep current permission checks server-side.
• Point eligible, exact rank outside top-k: query-document semantic mismatch, noisy chunk, unsupported language, poor chunking, wrong embedding model or query transformation. Improve representation, use alternate query formulation or lexical/hybrid recall; increasing k is a measured trade-off, not a guarantee.
• Point exact-rank high but absent from ANN: approximate-index, candidate budget, index optimization or quantization issue. Tune the engine on a held-out benchmark while monitoring latency and Recall@k.
• Point survives ANN but vanishes after filters or fusion: correct post-filtering, candidate pool size, rank fusion, grouping or deduplication logic.
• Point returned by a shard but absent globally: repair coordinator merge, shard routing, partial-results handling, replica freshness or timeout behavior.
• Point survives all candidates but not the final answer: reranker/context/LLM issue; keep this lesson focused on the first-stage visibility boundary.

4. Beginner-friendly numerical incident

• Build a labelled set of 200 production-like questions, each with one verified authorized answer-bearing chunk in the same snapshot. With first-stage top_k=10, the baseline retrieval finds the correct chunk for 136/200 = 68% Recall@10 (for one expected chunk per query, hit rate equals this recall definition).
• For the Singapore reimbursement query, direct ID lookup finds travel-2026-v3-chunk-43. Exact authorized search ranks it #4 of 50; the current HNSW call with hnsw_ef=16 and top-10 does not return it. This narrows the root cause to ANN candidate generation or an inconsistent serving snapshot—not an absent source document.
• A controlled same-snapshot experiment increases hnsw_ef from 16 to 128 (not a universally optimal setting). The chunk now appears at ANN rank #5. Across 200 queries, illustrative Recall@10 rises from 136/200 = 68% to 174/200 = 87%. Search p99 rises from 90 ms to 165 ms. Do not accept that trade-off without checking the SLO.
• Further inspection discovers a separate issue: 12 relevant chunks are candidates in dense search but dropped because a post-filter is evaluated against an incorrectly typed effective_year field. After repairing the metadata and moving to a supported pre-filtered search path, suppose the final benchmark reaches 186/200 = 93% Recall@10, with p99 175 ms.
• Relative gain = (93 − 68)/68 × 100 ≈ 36.8%; absolute gain = 25 percentage points. However, the corrected filter may legitimately exclude some other points; all tests must compare against the authorized relevance set.
• Hypothetical stage trace for this incident: source present=YES → chunk ID exists=YES → ACL eligible=YES → exact rank=4 → ANN missing → ANN tuned rank=5 → post-filter rejected due to schema bug → corrected filtered result rank=5. This is *illustrative*, not a claim that every missing-document incident has these two causes.

5. Engineering solution

• Establish a candidate provenance trace that records, for each test request: eligibility decisions, exact and ANN rank, per-shard IDs, branch candidate sets, fusion decisions, reranker input and final citation. Avoid logging raw confidential text; use protected IDs/hashes and short retention.
• Add an internal known-ID diagnostic tool that fetches and validates the expected point, computes its exact score/rank in a bounded authorized set, and compares to ANN results on the same snapshot.
• Repair the first failing stage with smallest blast radius: metadata fix, correct collection/named-vector route, replica consistency, targeted reindex, tuned ANN candidate budget or corrected fusion. Do not attempt all fixes simultaneously.
• For Qdrant, compare exact=true versus ANN with matching filter and limit; assess payload indexes, indexed_only visibility, HNSW parameters and filtered-query planner. For Elasticsearch, compare num_candidates and pre-filtering to post-filtering on the same vector and eligible corpus.
• For semantic misses where exact rank is low (numerically large), consider better answer-bearing chunks, title/header enrichment, query rewriting without losing intent, hybrid lexical retrieval, and a measured wider candidate pool.
• Run tests on realistic tenant, date and document-version filters; guard against exposing revoked sources in any diagnostic/fallback route. Roll out via shadow evaluation and canary traffic with rollback thresholds.
• Measure retrieval Recall@k, first-stage candidate presence, filter eligibility, latency, exact-vs-ANN gap, and relevant-citation rate, not only approximate similarity scores.

6. Trade-offs, false fixes, and failure scenarios

• Increasing top-k: can admit more relevant chunks but raises reranker cost/noise and does not recover a point omitted before candidate generation by an index or filter.
• Increasing HNSW ef or `num_candidates`: often improves ANN recall but consumes CPU, RAM and latency; evaluate production p99 and concurrency.
• Exact kNN: excellent diagnostic and sometimes viable for small filtered sets, but costly over millions of vectors.
• Removing metadata filters: may make a missing result appear yet break tenant, legal or source-version rules. Isolated authorized debugging may compare eligibility hypotheses; production must never bypass them.
• Hybrid recall: helps lexical matches and rare terms, but a bad fusion weight or overly small branch depth can still discard the correct candidate.
• Rebuilding the index: might fix a provable index-health defect, but cannot fix wrong embeddings, lost chunks, wrong filters, stale aliases or a truncating reranker.
• Self-query bias: using a document's own embedding to test discoverability verifies a lower-level path; it does not prove natural-language query relevance.
• Partial shard responses and deadlines: systems may return superficially plausible top-k without full shard coverage. Treat partial-results modes and completeness flags as first-class evidence.

7. Prevent recurrence

• Track stage-wise candidate survival on golden authorized queries: exists → eligible → exact top-k → ANN top-k → per-branch candidate → fusion output → reranker input → final cited source.
• Run nightly exact-vs-ANN comparisons on fixed, versioned query/corpus snapshots and monitor Recall@10 by tenant, filter class, shard, source version, query length and document age.
• Add deployment checks for alias/model changes, index optimizer states, named-vector selection, payload-field types, pre/post-filter planning, fusion weights, per-shard request completeness and candidate cutoffs.
• Add a debugger runbook with 10-minute triage: capture trace, fetch expected ID, validate filters, exact baseline, ANN comparison, fusion trace, remediate first failure, verify security and latency.
• Inject failure tests for missing chunk metadata, ACL revisions, stale replicas, narrow candidate budgets, OCR gaps, filter schema mismatch, partial shard errors and truncated reranker contexts.
• Monitor p95/p99 and cost alongside retrieval gains; require documented acceptance thresholds, canary rollouts and reversible configuration changes.

Interview takeaway: Start with a known answer-bearing chunk and locate the first boundary where it disappears: source and lineage → point existence → eligibility → exact rank → ANN candidates → per-shard merge → filtering/fusion → reranker. The correct fix depends entirely on that boundary.

References

• Qdrant, Search, exact search and indexed-only behavior: https://qdrant.tech/documentation/search/
• Qdrant, Payload filtering: https://qdrant.tech/documentation/search/filtering/
• Elasticsearch, exact and approximate kNN: https://www.elastic.co/docs/solutions/search/vector/knn
• Elasticsearch, filtered approximate kNN versus post-filtering: https://www.elastic.co/docs/solutions/search/vector/knn/filtered-knn-search
• Weaviate, vector and named-target search: https://docs.weaviate.io/weaviate/concepts/search/vector-search