Your assistant cites the correct document but makes a claim that the cited passage does not support. How would you detect this automatically?

Production incident (illustrative): A company procurement assistant answers, “For Singapore contractors, expenses above SGD 500 automatically qualify for finance approval within two days,” citing the *correct* 2026 Contractor Expense Policy. But the cited paragraph says “Claims above SGD 500 must be reviewed by the regional finance team; no automatic approval or fixed turnaround time is guaranteed.” The source document is correct, current, authorized and present in the prompt. The generated answer changes “review” to “automatically approved,” invents a two-day deadline, and drops an explicit denial. A document-level citation check passes while a claim-to-evidence entailment check would fail.

Key principle: A citation is an address, not proof. Distinguish citation validity (document/version exists and is authorized), passage applicability (the cited span governs this entity/date/region), and claim support (the exact cited span logically entails the statement in the response). A claim can be factually plausible—or even true in some other source—yet still be unsupported by its attached citation. Detect this at the smallest meaningful claim/span level, not only the full answer or document level.

1. Contain the incident and establish a reliable evidence trace

• For consequential finance, legal, safety or policy answers, gate unsupported factual claims before release. Either regenerate using verified evidence, remove the unsupported statement, or explicitly abstain; never retain a confident wrong citation because the document title looks relevant.
• Trace the user question, tenant/principal, active document version, retrieval snapshot, final model-visible spans, generated answer, citation markers and the exact passage text referenced by each citation. Record canonical source ID, version, section, offsets, content hash and effective date.
• Confirm retrieval and packing did not lose the clause. Q266 covers lost spans during context trimming; Q267 covers position effects despite spans being present. Here the supporting source is available, but the answer overclaims or contradicts the cited sentence.
• Validate that the citation resolver maps markers to the correct chunk and current version. A citation to a file's landing page or title is not enough for automatic support verification.
• Store only necessary, access-controlled evaluator inputs; document text and user questions may contain confidential material. Grounding checks must reuse the user's authorized evidence set, never search across another tenant to validate a claim.

2. Decompose the answer into atomic, testable claims

• Split compound sentences into discrete propositions without losing subjects, negation, modalities, quantities, and conditions. Example answer: “Claims over SGD 500 are automatically approved within two days.”
• Atomic claim A: Claims over SGD 500 are automatically approved. Atomic claim B: Approval is completed within two days. Optionally distinguish claim C (threshold SGD 500) from A because the source supports the threshold while denying automatic approval.
• Normalize referents using local question context: “they” means *Singapore contractors*; “it” means a reimbursement claim. Preserve identity, time range, jurisdiction and policy edition in each structured claim.
• Keep attribution: link each claim to the citation(s) attached to its original sentence; if citations are only listed at paragraph end, define a consistent conservative citation scope and mark ambiguous mapping for review.
• Exclude purely subjective style statements from factual groundedness denominators, but include legal/financial instructions and all implied factual commitments. Avoid artificially inflating faithfulness by splitting “SGD 500” into trivial supported fragments.
• Treat quoted text specially: verify byte/normalized-text agreement and source offsets for direct quotations, alongside entailment of surrounding interpretation. Exact quote matching alone does not validate an inference.
• Candidate claim schema: claim_id, answer_sentence_id, subject, predicate, object, qualifiers, time, region, modality, negation, citation_ids. Keep stable identifiers for regression evaluation and audit.

3. Resolve exact citation spans before judging entailment

• Use the same model-visible, authorized citation evidence attached to the response, not all text in the source document. This distinguishes “somewhere in the document” from “supported where cited.”
• Resolve each citation to source_id + version + section + [start,end) and verify the text hash against the evidence in the final prompt. Reject broken links, mismatched revisions and stale/unauthorized spans as invalid citation before attempting semantic support.
• Fetch a small bounded adjacent window only when the cited sentence refers to definitions, exceptions or tables immediately around it; maintain original provenance and permissions. Report separately whether the precise cited window versus expanded validated window supports the claim.
• For multi-hop or multi-citation claims, allow a bounded evidence set if the inference requires combining two passages. Record which claim is supported by which citations, and flag missing prerequisites. Do not declare an unsupported claim “supported” merely because some unrelated passage in the full retrieved corpus could conceivably support it.
• Check that evidence applies to the user's role, geography, date, product version and scope; an expired employee policy cannot support a current contractor instruction, even when wording is similar.

4. Automatic claim-to-evidence classification

• For each claim, cited_span, question_context pair run an evidence-conditioned NLI or LLM-as-judge check with three labels: SUPPORTED (entailed), CONTRADICTED (the evidence entails its negation), and INSUFFICIENT_EVIDENCE (neither support nor contradiction). A fourth INVALID_CITATION status comes from provenance resolution.
• Force judges to output minimal quoted supporting/conflicting evidence, exact offsets, preserved qualifiers and a structured label. Verify judge-provided quotes against the span rather than trusting fabricated quotations.
• Require conservative treatment of quantified and modal terms: “may,” “must,” “can request review,” “automatically approved,” “within,” “at least,” “up to,” “unless,” and “not” are materially different commitments.
• Use deterministic validation where possible: numeric threshold comparisons, ISO dates, currency/units, exact product codes, status enums, source version, and quote substrings. A semantic judge should not override a clearly conflicting structured value.
• For claim A, the policy says “must be reviewed” and “no automatic approval”. The proper classification is CONTRADICTED, not supported just because “SGD 500” and “finance” appear in the excerpt.
• For claim B, “no fixed turnaround time is guaranteed” contradicts a guaranteed two-day turnaround. If the cited passage simply omitted timing entirely, B would be INSUFFICIENT_EVIDENCE, not contradicted. The distinction matters for remediation.
• Never equate embedding cosine similarity or lexical overlap to logical entailment; two contradictory sentences can be lexically and semantically similar. A groundedness score at whole-answer level can mask one dangerous unsupported assertion.
• Use independent human-calibrated evaluation labels for ambiguous claims. LLM judges can be overconfident, sensitive to wording, or wrong about negation and arithmetic; do not describe an automated judge as perfect proof.

5. Root-cause debugging decision tree

• Citation marker resolves to an old/incorrect passage: citation formatter, source-version resolver, document relocation or span offset bug. Repair provenance first, not prompting.
• Marker resolves correctly but a key exception sentence was omitted from the final prompt: Q266 context loss; fix delivery.
• Claim supported elsewhere in the prompt, not by attached citation: citation-to-claim alignment defect. Move/add the correct precise citation only after verifying all constraints; otherwise remove the unsupported citation.
• Source and cited passage are correct but claim contradicts them: generation/reasoning issue, negation/modality misread, overgeneralization or blended memory. Reject/regenerate with exact span and stricter claim validation.
• Evidence is silent about claimed timing, quantity or authority: unsupported extrapolation. Seek an additional authorized authoritative source or express uncertainty; do not invent details.
• Claim combines facts from two policies without applicable join: multi-source reasoning and source authority failure. Confirm role, effective dates and policy precedence.
• Grounding judge reports failure on a true paraphrase: evaluation false positive; audit the judge rubric, necessary preceding definitions and local context. Re-evaluate using adjudicated examples.
• Judge reports support for a wrong number or negation: evaluation false negative; add structured equality/range tests and adversarial negative examples.
• Answer contains several facts but only one is unsupported: revise that claim rather than marking every supported claim invalid. However, for high-stakes operational actions a single critical unsupported claim may require withholding the entire answer.

6. Worked numerical production example

• Evaluate 200 answers containing 1,000 human-annotated factual claims. Gold labels: 850 supported, 90 contradicted, 60 insufficient evidence, so 150 genuinely unsupported or contradicted in total. Assume citation IDs and source versions are valid for this controlled test.
• Before the verifier, all 200 answers are returned. Suppose only 126/200 = 63% are completely claim-supported; this answer-level rate is a separate annotation and must not be inferred from claim counts alone.
• An automated claim checker flags 135 of the 150 truly non-supported claims (recall = 135/150 = 90%) and incorrectly flags 34 of the 850 supported claims (false-positive rate = 34/850 = 4%).
• Its precision on flagged claims is 135/(135+34) = 135/169 ≈ 79.9%; 15 truly unsupported claims remain unflagged. These numbers show why automatic detection should not be sold as perfect safety assurance.
• In a controlled remediation, take 169 flagged claims for targeted removal, regeneration with cited span, or human review depending on risk. Assume 120 of the 135 true positives are successfully repaired without introducing new errors. On the same 1,000-claim set, genuine unsupported claims fall from 150 to 30 (150-120=30), or 15.0% to 3.0%. Of the remaining 30, 15 were missed and 15 flagged but unrepaired.
• The 34 false positives should not be automatically deleted as “false facts”; route ambiguous or business-critical ones to review. Suppose rerun human grading confirms 178/200 = 89% fully supported answers, versus 126/200 = 63% baseline. That is a +26-point illustrative improvement, not an algebraic consequence of the claim repairs.
• A claim verifier adds p95 180 ms → 310 ms in this hypothetical setup, plus evaluation tokens. For a high-volume low-risk assistant, use risk-tiered checks; for approval-policy guidance, stricter verification may justify latency.
• These counts are illustrative, not live MachinoAI measurements. Real deployment must separately calibrate detection recall, precision, changes in answer completeness, abstentions, costs and customer outcomes.

7. Engineering solution and runtime workflow

• A. Produce citeable spans: attach stable source/version/section-offset IDs to the exact retrieved text used in the final prompt. Do not let the model invent source IDs or cite unseen document passages.
• B. Generate structured answer and provenance: request per-sentence citation IDs, then post-process claims and map them to supporting spans. A citation is only accepted after a trusted resolver validates its source.
• C. Decompose factual claims: use a lightweight constrained extractor, version its prompt/model, and test that no material assertion is lost or split incorrectly. Ensure negation, numerical comparisons and qualifiers survive decomposition.
• D. Verify claims: run deterministic rule checks for exact facts and an NLI/LLM judge for semantic entailment, including labels and source offsets. Where multiple citations are required, check the bounded combined evidence and record each bridge.
• E. Make risk-aware decisions: supported claims pass; contradicted claims block; insufficient claims trigger authorized evidence refetch, clarification, conservative rewrite or abstention; invalid citations trigger source-resolution repair. Escalate disputed high-impact facts.
• F. Repair safely: regenerate only affected spans with the original verified text, then re-verify repaired claims. Avoid infinite regeneration loops by limiting retries and refusing unsupported claims when a bounded repair fails.
• G. Log auditable artifacts: claim ID, original answer span, source passage hash/version, verifier model/version, label, measured confidence (if calibrated), deterministic checks, action and eventual human adjudication. Protect sensitive inputs and maintain retention limits.
• H. Canary with labelled evidence: compare claim-level support, contradiction escapes, false-positive rate, valid-citation rate, whole-answer support, missing-critical-fact rate, abstention, latency and cost. A system can become more “faithful” merely by answering nothing, so maintain answer utility and completeness measures independently.
• I. Handle policy change: invalidate/re-check cached source mappings and answers when citation source versions, ACLs or document effective dates change. Grounding against stale content is not sufficient.

8. Evaluation design, trade-offs and failure scenarios

• Groundedness vs factual truth: “true on the internet” is not equivalent to “entailed by this cited source.” The verifier is a citation-support guard, not a universal truth oracle.
• Precision vs recall: aggressive blocking catches more unsupported claims but can suppress correct paraphrases; permissive verification improves coverage but lets harmful claims through. Set thresholds by the cost of false approvals, not a single global target.
• Claim splitting: smaller atomic units improve localization but can erase conditions if extraction drops a shared negation or temporal scope. Keep a structured qualifier ledger and evaluate decomposition quality.
• Passage-level vs document-level: broad document text may contain contradictory revisions or exceptions, making a wrong citation seem acceptable. Prefer the actual cited span and explicit version authority.
• Multi-hop evidence: some claims require combining independent excerpts; scoring one passage alone can yield false unsupported labels. Bound permitted support sets to authoritative cited material.
• NLI/LLM judges: scalable but imperfect on numbers, modality, conditional logic, tables, multilingual text and domain jargon. Use calibrated human labels, deterministic guards and regression suites.
• Verifier feedback loops: if the same model generates and judges, shared biases may cause correlated errors. Compare independent verifier models and periodic human adjudication for critical slices.
• Self-repair: can improve support but also remove necessary qualifications or invent new citations. Always re-run support checks and preserve evidence trace.
• Abstention: necessary when the citation cannot substantiate a claim, but excessive abstention harms helpfulness; monitor supported answer completeness separately.
• Conflicting sources: selecting any text that entails the claim is unsafe when an authoritative current policy contradicts it. Enforce temporal, jurisdictional and document precedence.

9. Prevention checklist

• Maintain a golden claim-evidence dataset with entailed, contradicted, insufficient and invalid-citation examples. Include "same document but wrong paragraph" and "right quote but wrong effective date."
• Add adversarial transformations: reverse a negation, change 500→5,000, “may”→“must,” “review”→“automatically approve,” swap entities, change two days to ten, truncate an exception, and attach a real but irrelevant citation.
• Evaluate claim-level unsupported rate, contradiction escape rate, supported-citation precision, whole-answer all-claims-supported rate, citation resolution, false-positive rate, missed critical claims, completeness, costs and p95/p99.
• Segment by model, document version, answer length, citations per claim, domain, language, tables, multi-document questions and sensitive actions. Alert on increases after prompt, reranker, packing or citation-format changes.
• Audit the citation resolver separately from semantic verification. Ensure relocated files preserve stable IDs, spans, current ACLs and source versions.
• For high-risk answers, sample daily human audits and feed adjudicated failure cases into the regression suite. Recheck labels after source updates rather than treating old judgments as timeless ground truth.
• Fail closed when a cited source is forbidden or missing; never widen retrieval across tenants to make a claim appear supported.
• Require a bounded regeneration budget and clear rollback criteria if an updated verifier increases false blocks, omissions or latency.

Interview takeaway: Detect “right document, wrong claim” by linking each atomic claim to the exact cited, authorized passage and checking entailment, contradiction or insufficient evidence. Validate citation provenance first, protect numeric/negative qualifiers, measure verifier precision/recall on human labels, and reverify every repaired claim. A clickable citation is *not evidence that the claim is grounded*.

Research references

• Es et al., *RAGAS: Automated Evaluation of Retrieval Augmented Generation* (EACL 2024): https://aclanthology.org/2024.eacl-demo.16/
• Min et al., *FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation* (EMNLP 2023): https://aclanthology.org/2023.emnlp-main.741/
• Microsoft Foundry, RAG evaluators: https://learn.microsoft.com/en-us/azure/foundry/concepts/evaluation-evaluators/rag-evaluators
• Microsoft Azure, *Large Language Model End-to-End Evaluation Phase*: https://learn.microsoft.com/en-us/azure/architecture/ai-ml/guide/rag/rag-llm-evaluation-phase