What is hallucination in an LLM?

Hallucination in a Large Language Model (LLM) happens when the model produces information that sounds plausible but is false, fabricated, or unsupported by the evidence it is supposed to use.

Unlike a simple spelling mistake, a hallucination can look like a confident, well-written answer. The problem is reliability of the content, not necessarily grammar or fluency.

Beginner-friendly example

Imagine a company's official policy says:

> Customers may request a refund within 30 days of purchase.

A user asks an LLM: "How long do I have to request a refund?"

• Grounded answer: "You can request a refund within 30 days."
• Hallucinated answer: "You have 60 days, and the policy was updated last month."

The second answer contradicts the document and invents an update. The model may state it confidently even though no supplied evidence supports it.

Another example is an LLM inventing a research-paper title, citation, URL, or API method that does not exist.

How can you recognize a hallucination?

Common signs include:

• False factual claims: An incorrect date, figure, person, scientific statement, or event presented as true.
• Fabricated details: Nonexistent references, quotations, products, policy clauses, or sources.
• Contradictions with supplied context: A document says one thing, but the answer reports another.
• Unsupported assertions: The model presents a claim as established even though the information it was instructed to rely on does not support it.

Important distinction: An answer that goes beyond the supplied document is not necessarily false. It may be correct using outside knowledge. But if the task explicitly requires *only* document-supported claims, it is ungrounded and can be treated as a faithfulness failure. Whether it is factually false requires a separate check.

Researchers use somewhat different hallucination definitions and taxonomies, so always specify what your evaluation considers an error.

Why can a model produce convincing false information?

A generative LLM predicts plausible next tokens using patterns it learned and the context it receives. Fluent generation is not a built-in guarantee of factual verification.

Hallucinations can surface when a model lacks reliable information, encounters conflicting evidence, misunderstands its context, or continues generating beyond what its evidence supports. The detailed mechanisms belong to the next question: "Why do Large Language Models hallucinate?"

Hallucination vs. other behavior

• Creative fiction: Invented characters in a requested story are not a hallucination, because factual reporting was not the goal.
• A stated guess: "I'm not sure; it might be..." can still be wrong, but it signals uncertainty rather than disguising the guess as verified fact.
• A refusal or abstention: "The document does not specify this" is often the appropriate answer when no reliable source is available.
• A hallucination: A made-up claim is delivered as if it were supported or true.

How do you measure hallucinations?

For an initial application-level check:

1. Build a held-out set of factual questions with reliable evidence.
2. Generate answers using a fixed model and prompting setup.
3. Have reviewers check every material claim against the evidence.
4. Define and count answers containing at least one hallucinated or unsupported claim.

Example: If 3 out of 20 answers have at least one unsupported material claim:

Observed answer-level hallucination rate
  = 3 / 20 × 100
  = 15%

This describes that small test sample. A claim-level error rate would have a different denominator; neither result can be generalized without sufficiently representative data and careful review.

How can hallucinations be reduced?

• Use authoritative context or retrieval (RAG) when answers depend on external or changing information.
• Require evidence for factual claims and check that citations actually support them.
• Allow the model to say "I don't know" when evidence is insufficient.
• Validate high-stakes answers with deterministic checks, trusted tools, or human review.
• Evaluate on real application questions and track errors after model or prompt changes.

These measures can reduce some hallucinations, but none guarantees that every generated answer is correct. Even retrieval-backed systems can misread documents or invent unsupported conclusions.

In short: An LLM hallucination is a plausible-looking but false or ungrounded generation in a task that requires factual or evidence-faithful output. Check claims against evidence rather than trusting the confidence or polish of the writing.

References: Survey on Hallucination in Large Language Models, HalluLens: LLM Hallucination Benchmark, and Survey and analysis of hallucinations in LLMs.