Large Language Models

Large language models are deep learning models, usually transformer-based, trained to predict tokens from large corpora. After pretraining, they are often instruction-tuned and aligned so they can follow prompts and assist users.

What is SIMD (Single Instruction, Multiple Data) and how is it used in LLM inference?

SIMD (Single Instruction, Multiple Data) is a way for a processor to apply one instruction to several data values at once using vector registers. In LLM inference, SIMD helps accelerate repetitive numerical work—especially CPU based tensor operations such as dot products, elementwise activation functions, and operations involving quantized weights. How does SIMD work? Without SIMD, a scalar instruction might add two numbers at a time. With SIMD, one vector add instruction operates on several pairs of numbers held in different lanes of a vector register. SIMD does not automatically make an entire program four times faster: loads, stores, instruction overhead, dependencies, and memory bandwidth still matter. Numerical example: CPU vectorized LLM computation Suppose a CPU kernel multiplies 32 FP32 weights by 32 FP32 activation values , then sums the products for a dot product. A scalar approach performs 32 individual element multiplications before accumulation. A 256 bit AVX capable vector register can hold eight 32 bit floats . The kernel can therefore multiply the 32 pairs using four eight lane vector multiply operations , followed by vector accumulation and a final reduction. This counts the vector multiply groups—not every machine instruction required for a complete dot product. A real kernel also loads data, adds partial sums, and handles edge cases. Where is SIMD useful for LLM inference? Matrix and vector arithmetic: CPU implementations can vectorize inner loops of linear layers, dot products, and embeddings. Quantized inference: Optimized routines unpack or convert low bit weights and compute vectorized integer/floating point dot products, where supported by the CPU. Elementwise kernels: Operations such as activation functions, normalization steps, and scaling can process several elements per instruction. Sampling and preprocessing: Some repetitive token processing and probability array operations may benefit when their loops vectorize cleanly. Practical example: The open source llama.cpp inference engine supports CPU instruction sets such as AVX, AVX2, AVX 512 on x86 and Arm NEON . These backends help make running smaller or quantized LLMs on CPUs practical. What SIMD instructions exist? x86 AVX / AVX2: 256 bit vector registers; a vector can hold eight FP32 values. x86 AVX 512: 512 bit vector instructions, which can process up to sixteen FP32 lanes where an appropriate instruction and CPU support exist. Arm NEON: Commonly uses 128 bit vectors, such as four FP32 lanes. Arm SVE: Supports scalable vector lengths, enabling more portable vector length agnostic loops. The actual speed depends on the instruction type, CPU model, register pressure, memory layout, and arithmetic precision . Is SIMD the same as GPU parallelism? Not exactly. SIMD: One CPU style vector instruction operates across multiple data lanes. SIMT (Single Instruction, Multiple Threads): NVIDIA GPUs schedule groups of threads called warps ; threads follow a common instruction stream but have their own registers and can diverge on branches. Tensor Cores / matrix engines: Specialized hardware accelerates matrix operations; it should not be described as simply "more SIMD lanes." SIMD and SIMT are related data parallel ideas, but their programming and execution models are different. SIMD vs. fused kernels SIMD makes one operation act on several elements in parallel. Kernel fusion combines different operations to avoid launches or intermediate memory transfers. An optimized LLM kernel may benefit from both . When does SIMD provide little benefit? Poorly contiguous data, unpredictable branches, tiny vectors, reductions, or memory bandwidth bottlenecks can limit gains. Some compilers auto vectorize suitable loops; custom intrinsics are not always necessary. Benchmark end to end tokens/s, TTFT, TPOT, and the relevant kernels on the hardware you deploy. In short: SIMD speeds up LLM inference by applying CPU vector instructions to multiple tensor elements at once. Its largest gains come from well optimized numerical inner loops, not from treating every LLM operation as automatically parallel. References: Intel instruction set extensions, Arm Neon, NVIDIA SIMD vs. SIMT explanation, and llama.cpp.

What is workload-conditioned operator optimization in LLM inference?

Workload conditioned operator optimization means choosing or tuning the implementation of an LLM computation according to the workload it actually receives —such as tensor shapes, batch size, prompt/decode phase, data type, GPU architecture, and memory limits—rather than forcing every request through the same kernel configuration. Terminology: This is a broad optimization strategy, not one standardized algorithm or a requirement that the LLM itself make a routing decision. Why does one kernel not fit every LLM workload? An LLM repeatedly performs operations such as matrix multiplication ( GEMM ), attention, and normalization. Their dimensions and bottlenecks change across requests: Prefill: A prompt may contain hundreds of tokens processed together, creating larger matrix operations that can use GPU compute efficiently. Decode: A request usually adds one token per generation step. At low concurrency, the resulting matrix operations can be much thinner; launch overhead and memory movement may matter more. Batching and context length: Concurrent sequences and KV cache sizes change which operations dominate time and memory use. A kernel tuned for a large matrix may waste work or resources on a very small one. Practical example: selecting a GEMM kernel Consider an LLM projection represented as: Here, M is the number of token representations being multiplied in that operation. Case A — one token decode: M = 1. This is a thin matrix multiplication, so a kernel optimized for small M or matrix vector work may be preferable. Case B — prompt prefill: M = 512. A tiled matrix multiplication kernel with more parallel work may make better use of the GPU. Both cases compute the same mathematical operation . What changes is how it is executed . The correct kernel cannot be determined from M alone: GPU model, precision, tensor layout, batch size, and memory behavior also matter. How is the optimization performed? 1. Identify workload classes: Measure commonly encountered shapes, precisions, and devices. 2. Prepare candidate implementations: For example, different GEMM tile sizes, warp counts, pipeline stages, or compatible kernel backends. 3. Benchmark candidates: Compare warmed up latency and memory use for each representative workload. 4. Choose a configuration: Use a shape specific lookup, an optimized execution profile, or cached autotuning results. 5. Validate and monitor: Confirm numerical correctness and benchmark end to end serving metrics such as TTFT, TPOT, throughput, and p95 latency. What does autotuning look like? The Triton GPU programming framework supports @triton.autotune: it benchmarks several triton.Config alternatives, such as different matrix tile sizes and warp counts, with a key like ['M', 'N', 'K'] to distinguish shapes. This does not mean a production server should benchmark kernels on every user request. Autotuning and compilation can be expensive; tune representative workloads offline or cache results, then dispatch cheaply at runtime. Similarly, NVIDIA TensorRT supports optimization profiles for ranges of input dimensions and can select suitable implementation tactics while building an engine. How does it differ from related optimizations? Kernel fusion: Combines multiple compatible operations to reduce launches and intermediate data transfers. Workload conditioned optimization chooses which implementation or settings to use for a given workload. SIMD/SIMT: Hardware execution models that a selected kernel may exploit; they are not themselves workload selection policies. Conditional Split K: A more specific GEMM technique that selectively parallelizes work along the reduction dimension K when doing so benefits the matrix shape. It will be covered separately. What are the trade offs? More kernel variants increase tuning time, compiled code size, and maintenance. Selecting a suboptimal variant or repeatedly switching shapes/profiles can increase latency. Performance may also regress when the production mix differs from the tuning data. Always validate accuracy and measure whole service performance , not just a single microbenchmark. In short: Workload conditioned operator optimization uses shape, hardware, and execution context to pick efficient operator implementations. It is especially useful because prefill, decode, and concurrent serving stress GPU kernels differently. References: Triton matrix multiplication/autotuning tutorial, Triton autotune API, NVIDIA TensorRT dynamic shapes and optimization profiles, and Hugging Face tuned kernel configurations.

What is Conditional Split-K, and how does it optimize matrix operations?

Conditional Split K is a GPU matrix multiplication optimization that uses Split K only when the matrix shape and hardware make it worthwhile . Split K divides the reduction dimension K into independent pieces, computes partial matrix products in parallel, and combines them to form the final output. Terminology: Split K is an established GEMM technique. Conditional Split K describes a selection policy —use ordinary GEMM for some shapes and Split K for others—not a separate standardized mathematical operation. Why would a matrix multiplication need Split K? For the matrix product C = A × B : A has shape M × K . B has shape K × N . C has shape M × N . In a conventional tiled GEMM, GPU thread blocks divide work mainly across M and N . When these dimensions produce too few output tiles , many GPU multiprocessors can remain underused even if K is very large . Split K creates additional parallel work by assigning different K ranges to different thread blocks. This is especially useful for some skinny matrix multiplications in low batch LLM decoding. Beginner friendly numerical example Suppose an LLM projection has: Without Split K, each output element accumulates products along all 4,096 positions of K. With four K partitions , the GPU computes: Each partial result has shape [32, 128] . Instead of waiting for one output tile to finish the full K reduction, several GPU blocks may compute partial results concurrently. Important: Four partitions do not promise a 4× speedup. The hardware still performs approximately the same multiplication work plus a reduction of partial outputs, and extra memory traffic may offset the increased parallelism. What makes it “conditional”? A kernel dispatcher, heuristic, or autotuner may choose: Standard GEMM when M and N already produce enough output tiles to occupy the GPU, or the K dimension is too short. Split K GEMM when output tile parallelism is low and K is long enough to provide useful additional work. A tuned split factor (for example, 2, 4, or 8) rather than always splitting the same number of ways. The decision depends on M, N, K, tile sizes, data type, hardware, reduction overhead, and available workspace . There is no universally correct threshold . How does the GPU combine partial results? For parallel Split K , implementations commonly write partial sums to a workspace and then reduce them, sometimes using a separate kernel. Other designs use coordinated or atomic accumulation. The extra reduction means Split K may require: More intermediate memory. Additional writes and reads. Another kernel launch or synchronization. A different floating point addition order, so results should agree within numerical tolerances but are not guaranteed bit for bit identical . For the example above, four FP32 partial outputs contain 4 × 32 × 128 × 4 = 65,536 bytes (64 KiB) of intermediate values before considering any implementation specific overhead. When is it useful for LLMs? Small batch decode time linear projections , particularly with skinny activation matrices and large weight matrices, are plausible candidates. In research on W4A16 quantized LLM inference , a fused dequantization and GEMM kernel using Split K improved selected skinny matrix workloads on NVIDIA GPUs. Those reported gains are benchmark specific , not universal guarantees. Split K is not KV cache splitting, tensor parallelism across GPUs, or a change to the model's attention architecture. How should you evaluate it? Benchmark the same GEMM shapes with ordinary GEMM and several Split K factors. Record kernel latency, reduction overhead, temporary memory, numerical error, and performance across prefill versus decode workloads. Finally, confirm that any kernel level improvement benefits end to end TTFT, TPOT, or throughput . See “What is workload conditioned operator optimization in LLM inference?” for the broader strategy of selecting kernels based on workload. In short: Conditional Split K selectively parallelizes a long GEMM reduction dimension when the ordinary output tile grid leaves GPU resources idle— but skips Split K when the extra reduction costs more than it saves . References: NVIDIA CUTLASS: Parallelized reductions, Accelerating a Triton Fused Kernel for W4A16 Quantized Inference with SplitK, and CUTLASS Stream K / Split K scheduling example.

How are Large Language Models evaluated?

Large Language Models (LLMs) are evaluated by testing their responses on carefully chosen tasks and measuring how well they satisfy defined quality, reliability, and safety criteria. There is no single score that captures whether an LLM is good for every use case. A model may answer general knowledge questions well but perform poorly at following instructions, avoiding unsupported claims, or handling a company's specialized documents. What do we evaluate? Task correctness: Does the answer solve the problem? For example, exact match accuracy on questions with unambiguous answers or passing unit tests for generated code. Instruction following: Does the model obey constraints such as a required JSON schema, word limit, or requested format? Factuality and grounding: Are statements supported by reliable evidence or supplied context? An answer can be fluent yet incorrect. Reasoning and robustness: Does performance hold across difficult, reworded, or unfamiliar examples? Safety and fairness: Does the model avoid harmful outputs and perform acceptably across relevant user groups? Efficiency: How much latency, memory, and serving cost does the required quality entail? These are important complementary measurements, not substitutes for answer quality. Main evaluation methods 1. Automatic evaluation with reference answers: Compare predictions against known labels or expected outputs. Measures can include accuracy , exact match, or task specific scores. Exact string matching is often too strict for open ended writing. 2. Human evaluation: Trained reviewers score answers with a clear rubric—such as correctness, usefulness, or clarity—or choose the better of two responses. Multiple reviewers help reveal disagreement. 3. LLM as a judge: Another model grades or compares responses using a fixed rubric. This is scalable, but judges can show position bias, verbosity bias, and inconsistent preferences . Calibrate them against human reviewed examples. 4. Real world/task based evaluation: Test workflows the application actually needs, including tool use, structured output, failure handling, and user satisfaction. Public benchmark results alone cannot replace this. 5. Adversarial and regression testing: Include edge cases, misleading prompts, policy sensitive requests, and examples the previous version handled correctly. Simple numerical example Suppose an LLM is being considered for an internal support assistant. On 100 held out, manually checked questions , it produces 84 correct answers : Separately, among 50 answers requiring a cited company policy , reviewers find 6 answers containing an unsupported claim : Both measurements are useful. 84% accuracy does not mean the model is safe or sufficiently grounded , and the 12% estimate describes only that particular test set. The denominator, rubric, and data selection matter. What is an LLM benchmark? A benchmark is a predefined evaluation dataset and scoring procedure. Examples include: MMLU / MMLU Pro: Broad knowledge and reasoning questions. GPQA: Challenging expert written science questions. IFEval: Verifiable instruction following constraints. HumanEval: Code generation judged by executable tests. Different benchmarks test different abilities . Comparing two model scores is meaningful only when the dataset version, prompting format, few shot examples, answer extraction, and grading procedure are compatible. A practical evaluation workflow 1. Define the intended use case and what counts as success or failure. 2. Build separate development and held out test sets representing real requests, difficult edge cases, and important subgroups. Keep test examples out of training and prompt tuning. 3. Choose relevant metrics and rubrics. Prefer several metrics rather than one headline score. 4. Run each candidate model under controlled conditions: same prompts, tools, decoding setup, and scoring rules; record model and dataset versions. 5. Review failures: Classify factual errors, instruction violations, refusals, unsafe outputs, and brittle behavior. Manually audit a sample of automatically graded answers. 6. Compare against the existing model or baseline , including uncertainty or variability where practical, before deciding whether to deploy. Frameworks such as EleutherAI's LM Evaluation Harness and Stanford's HELM help standardize tasks and reporting. Common mistakes to avoid Benchmark contamination: A model may have encountered test examples during training, inflating scores. Overfitting the test set: Repeatedly tuning prompts on the final test destroys its independence. Trusting a single judge: An LLM judge is not a substitute for verified labels and human calibration. Ignoring distribution shift: A public benchmark may not resemble production documents or users. Conflating quality and speed: TTFT, TPOT, and tokens/s measure inference performance , covered separately in “How do you benchmark and profile LLM inference performance?” In short: Evaluate LLMs using representative tasks, reproducible measurements, human judgment, safety checks, and real world failure analysis . The best LLM is the one that meets the application's quality and operational requirements—not necessarily the one topping a public leaderboard. References: Holistic Evaluation of Language Models (HELM), Stanford HELM documentation, EleutherAI LM Evaluation Harness, and Quantifying Biases in LLM as a Judge Evaluations.

What is an LLM benchmark?

An LLM benchmark is a standardized set of tasks, test examples, and scoring rules used to measure a language model's ability on a particular capability. It makes model comparisons more repeatable than asking each model a few informal questions. For example, a benchmark might test science reasoning , instruction following , or code generation . A high score on one benchmark does not prove that a model is best at every task. What makes up a benchmark? 1. Dataset: Questions, prompts, or problems representing the target capability. Some datasets include reference answers, labels, or unit tests. 2. Task format: How the model is asked to respond—multiple choice selection, free form answer, code completion, or other output. 3. Evaluation protocol: The exact prompt template, whether examples are shown beforehand ( zero shot vs. few shot ), decoding settings, and how responses are extracted. 4. Scoring metric: A rule such as accuracy, exact match, test case pass rate, or an explicitly defined human/judge rubric. 5. Reported results: Scores plus details needed for comparison, ideally including uncertainty, dataset version, and model configuration. Important: A benchmark is not just a collection of questions. The scoring and evaluation protocol are part of its definition. Beginner friendly example Consider a reasoning benchmark with 100 questions . Model A gets 78 correct, and Model B gets 84 correct under the same scoring rules: Model B performed better on these 100 questions . This does not guarantee that it will outperform A on unseen questions, production tasks, or another benchmark. With only 100 examples, the apparent gap may also have substantial statistical uncertainty . Common examples of LLM benchmarks MMLU / MMLU Pro: Broad academic knowledge and reasoning across many subjects. GPQA: Difficult graduate level science questions. IFEval: Checks compliance with verifiable instructions , such as formatting constraints. HumanEval: Evaluates generated code by running tests for programming problems. Each benchmark measures a narrower capability than overall model quality. Multiple benchmarks are usually needed to understand a model's strengths and weaknesses. Benchmark vs. evaluation vs. leaderboard Benchmark: The standardized task set and evaluation procedure . Evaluation: The broader process of deciding whether a model meets your needs; it can include benchmarks, human reviews, safety tests, and internal datasets. Leaderboard: A ranking or comparison of models using benchmark results. Its meaning depends on the specific scoring rules and submissions. See "How are Large Language Models evaluated?" for the complete evaluation workflow. How do you run a benchmark? Open source tools such as EleutherAI's LM Evaluation Harness provide standard task implementations. After installing and configuring the harness, an illustrative command is: This evaluates the specified model on a named task and saves results. It is a demonstration command , not a claim about GPT 2's quality or an endorsement of HellaSwag as a complete evaluation suite. For a fair comparison, keep the benchmark version, model prompt/chat template, zero /few shot settings, response extraction, and metric consistent. A model scored with five examples in its prompt should not be compared directly against a zero shot result without explaining the difference. What are the limitations? Data contamination: Benchmark examples appearing in training data can inflate scores. See "What is benchmark contamination in LLMs?" Benchmark overfitting: Repeated optimization against a public test may make the model good at that test rather than at the underlying capability. Prompt sensitivity: Different instructions, chat templates, or answer extraction rules can change scores. Coverage gaps: A knowledge benchmark might reveal little about factual grounding, production tool use, safety, or performance in a specific language. Small score differences: Differences can be affected by sample size, sampling variability, or evaluation setup. Best practice: Treat public benchmarks as one signal , then test your LLM on a separate set of realistic application tasks and inspect failure examples. In short: An LLM benchmark provides a shared test and measurement procedure for comparing models on a defined capability—not a universal measure of intelligence. References: EleutherAI LM Evaluation Harness, Harness evaluation CLI and task settings, Stanford HELM, and Measuring Massive Multitask Language Understanding.

What is perplexity in language models?

Perplexity (PPL) measures how well a language model predicts the actual next tokens in a text dataset . It is a common evaluation metric for autoregressive (causal) language models , which assign a probability to each next token given the preceding context. Lower perplexity generally means better next token prediction on that same test data and under the same evaluation setup. It does not automatically mean better reasoning, factuality, or conversational usefulness. What does perplexity mean intuitively? Imagine an LLM reading: A model that assigns a high probability to the actual next token (for example, a token representing "Paris") is less surprised by that observation than a model assigning it a very low probability. Perplexity summarizes this predictive surprise across many held out tokens , rather than evaluating just one sentence. Simple numerical example Suppose we evaluate two toy models on the same tokenized test sequence : Model A assigns probability 0.5 to every actual next token. Model B assigns probability 0.25 to every actual next token. Their perplexities would be: Model A is better at predicting this sequence , because it consistently gives the observed tokens greater probability. The example deliberately uses constant probabilities for intuition; real token probabilities vary. A perplexity of 1 represents ideal certainty about every observed next token. Valid perplexity is at least 1 . It is not a percentage, and a result of 4 does not mean 4% error. What is the mathematical idea? Perplexity is the exponential of average negative log likelihood (cross entropy loss) on the scored tokens: For token positions 1 through N, this is commonly written as: The model is evaluated against the actual reference tokens , using preceding reference context. It does not need to generate a paragraph freely and compare the final prose with an ideal answer. The separate question "How is perplexity calculated for an LLM?" will cover the full procedure and token level arithmetic. Where is perplexity useful? Comparing checkpoints of the same model family on an identical held out corpus. Tracking language model training or fine tuning , alongside validation loss. Measuring adaptation to a specific text domain , such as medical or legal documents. Diagnosing distribution shift , when text from a new domain is harder for the model to predict. For example, if a checkpoint's perplexity decreases on a fixed validation corpus, its predictive fit to that corpus has improved. That alone does not prove its downstream task accuracy has improved. Important limitations Tokenization matters: Different tokenizers may split the same sentence into different token sequences, so their token level perplexity values are not directly comparable without careful normalization and methodology. Dataset matters: Different corpora, preprocessing, context lengths, and evaluation settings produce different numbers. Context window effects: For finite context models, splitting long texts into isolated chunks discards useful context. A strided sliding window evaluation can give a more faithful estimate, and may report a lower perplexity than disjoint chunks. It measures prediction, not truth: A model may assign high probability to fluent but false or unsafe text. Not a universal model metric: Standard autoregressive perplexity is not directly defined in the same way for masked language models such as BERT. Perplexity vs. accuracy Accuracy: Often checks whether the model chose the correct label or answer. Perplexity: Measures the probabilities assigned to all observed next tokens , even when the model's top predicted token is different. Two models can achieve the same accuracy on a benchmark while having different perplexity scores on a text corpus. In short: Perplexity tells us how predictable real text is to a causal language model . Use it on consistent, held out text with a clearly documented tokenizer and context evaluation protocol—and combine it with task based evaluation before choosing a production LLM. Reference: Hugging Face — Perplexity of fixed length models.

How is perplexity calculated for an LLM?

Perplexity (PPL) for a causal LLM is calculated by measuring how much probability the model assigns to the actual next token at each scored position , averaging those negative log probabilities, and taking the exponential . The formula is: Here N is the number of valid predicted tokens , not the number of sentences or the vocabulary size. Logarithms are natural logs. For the intuition behind this metric, see “What is perplexity in language models?” Step by step numerical example Suppose an LLM predicts three observed tokens and assigns these probabilities to the tokens that actually occur : Token 1: 0.50 Token 2: 0.25 Token 3: 0.125 Step 1 — Take the negative natural log of each probability: Step 2 — Average the losses across the three scored tokens: Step 3 — Exponentiate the average: Result: Perplexity ≈ 4. Lower loss produces lower perplexity. Do not average the probabilities first and invert the result; that is generally a different calculation. The example assumes preceding context exists for each token. In a real model, the first input token usually has no preceding token to predict it unless a beginning of sequence token is provided. Where do the token probabilities come from? 1. Tokenize a held out evaluation text with the model's tokenizer. 2. Run the model using the reference tokens as preceding context (often called teacher forcing ). 3. For each scored position, obtain the model's next token probability distribution . 4. Select the probability assigned to the reference next token , not necessarily the highest probability token. 5. Sum its negative log probabilities, divide by the number of valid scored tokens, and exponentiate. You do not need the model to freely generate an answer and then compare the generated paragraph to the reference. Practical Python example with Hugging Face This example calculates perplexity for one short text using a causal LM such as GPT 2: For GPT 2 and similar Hugging Face causal LM classes , passing labels=input ids invokes the next token label shift inside the model . outputs.loss is the mean cross entropy over the predicted positions, and torch.exp(outputs.loss) gives their perplexity. This is a demonstration for a short text, not a reliable benchmark by itself. How do you calculate corpus level perplexity correctly? A serious evaluation uses many held out tokens. Track: Do not simply average perplexity values from different chunks , particularly if they contain different numbers of valid tokens. First aggregate token weighted losses , then exponentiate once. Ignore padding or context only positions in the loss. Hugging Face models commonly use label value 100 to mask ignored positions. When a causal model internally shifts labels, count only the labels that actually contribute to its returned loss. What if the text exceeds the context window? For fixed context Transformers, evaluating disjoint chunks independently loses preceding context at every new chunk boundary. A strided sliding window is usually a better approximation: Reuse overlapping context from the previous window. Calculate loss only for the new target tokens , not the overlapping context tokens. Sum those losses and divide by the true number of predicted tokens. This may produce a different—often lower—perplexity from disjoint chunking because each target token gets more relevant context. Common mistakes Incorrect token alignment: Predict token i+1 using logits from position i, not logits from the same target position. Wrong denominator: Divide by scored tokens, excluding padding and other masked positions. Comparing different tokenizers: Token level PPL is sensitive to tokenization and test preprocessing. Mixing evaluation protocols: Keep the test corpus, context window/stride, model mode, and prompt formatting consistent. Interpreting PPL as accuracy: Low perplexity measures predictive fit, not guaranteed factual correctness or reasoning quality. In short: Get the probabilities of the actual next tokens , calculate their mean negative log likelihood , and apply exp . For valid model comparisons, also use consistent tokenization and context window evaluation. References: Hugging Face: Perplexity of fixed length models and Hugging Face: Causal language modeling.

What is hallucination in an LLM?

Hallucination in a Large Language Model (LLM) happens when the model produces information that sounds plausible but is false, fabricated, or unsupported by the evidence it is supposed to use . Unlike a simple spelling mistake, a hallucination can look like a confident, well written answer. The problem is reliability of the content , not necessarily grammar or fluency. Beginner friendly example Imagine a company's official policy says: Customers may request a refund within 30 days of purchase. A user asks an LLM: "How long do I have to request a refund?" Grounded answer: "You can request a refund within 30 days." Hallucinated answer: "You have 60 days, and the policy was updated last month." The second answer contradicts the document and invents an update. The model may state it confidently even though no supplied evidence supports it. Another example is an LLM inventing a research paper title, citation, URL, or API method that does not exist. How can you recognize a hallucination? Common signs include: False factual claims: An incorrect date, figure, person, scientific statement, or event presented as true. Fabricated details: Nonexistent references, quotations, products, policy clauses, or sources. Contradictions with supplied context: A document says one thing, but the answer reports another. Unsupported assertions: The model presents a claim as established even though the information it was instructed to rely on does not support it. Important distinction: An answer that goes beyond the supplied document is not necessarily false . It may be correct using outside knowledge. But if the task explicitly requires only document supported claims, it is ungrounded and can be treated as a faithfulness failure. Whether it is factually false requires a separate check. Researchers use somewhat different hallucination definitions and taxonomies, so always specify what your evaluation considers an error. Why can a model produce convincing false information? A generative LLM predicts plausible next tokens using patterns it learned and the context it receives. Fluent generation is not a built in guarantee of factual verification. Hallucinations can surface when a model lacks reliable information, encounters conflicting evidence, misunderstands its context, or continues generating beyond what its evidence supports. The detailed mechanisms belong to the next question: "Why do Large Language Models hallucinate?" Hallucination vs. other behavior Creative fiction: Invented characters in a requested story are not a hallucination , because factual reporting was not the goal. A stated guess: "I'm not sure; it might be..." can still be wrong, but it signals uncertainty rather than disguising the guess as verified fact. A refusal or abstention: "The document does not specify this" is often the appropriate answer when no reliable source is available. A hallucination: A made up claim is delivered as if it were supported or true. How do you measure hallucinations? For an initial application level check: 1. Build a held out set of factual questions with reliable evidence. 2. Generate answers using a fixed model and prompting setup. 3. Have reviewers check every material claim against the evidence. 4. Define and count answers containing at least one hallucinated or unsupported claim. Example: If 3 out of 20 answers have at least one unsupported material claim: This describes that small test sample . A claim level error rate would have a different denominator; neither result can be generalized without sufficiently representative data and careful review. How can hallucinations be reduced? Use authoritative context or retrieval (RAG) when answers depend on external or changing information. Require evidence for factual claims and check that citations actually support them. Allow the model to say "I don't know" when evidence is insufficient. Validate high stakes answers with deterministic checks, trusted tools, or human review. Evaluate on real application questions and track errors after model or prompt changes. These measures can reduce some hallucinations, but none guarantees that every generated answer is correct . Even retrieval backed systems can misread documents or invent unsupported conclusions. In short: An LLM hallucination is a plausible looking but false or ungrounded generation in a task that requires factual or evidence faithful output. Check claims against evidence rather than trusting the confidence or polish of the writing. References: Survey on Hallucination in Large Language Models, HalluLens: LLM Hallucination Benchmark, and Survey and analysis of hallucinations in LLMs.

Why do Large Language Models hallucinate?

Large Language Models hallucinate because generating a plausible next token is not the same as verifying a claim against reality. Errors can arise from the training data , the learning objective , incentives that favor answering over admitting uncertainty, and what happens when the model uses context or generates a response . No single cause explains every hallucination. Researchers commonly group the causes into data, training, and inference factors. 1. Incomplete, incorrect, or outdated training data An LLM learns patterns from data, not from a complete, constantly updated fact database. Missing knowledge: Rare facts may appear too few times for the model to learn them reliably. Conflicting sources: Training data may contain contradictory names, dates, or claims. Outdated information: A policy, product feature, or public fact may change after the relevant training data was collected. When asked about a fact it does not reliably know, the model can still produce a plausible looking answer. 2. Next token prediction does not directly check truth During typical causal language model pretraining, a model learns to assign high probabilities to tokens that follow earlier tokens in its data . For instance, given: the model may produce a likely looking year . But selecting a probable continuation does not mean it has looked up the paper or verified its publication date. Important: The next token objective is not the only reason for hallucination. Models can learn many true facts from this training, and later training can improve accuracy. The mismatch matters especially for uncertain or poorly represented claims. 3. Training and evaluation can reward guessing Many question answer evaluations use a simple rule: correct answers get credit, while both wrong answers and "I don't know" get no credit . Imagine a model has only a 25% chance of guessing correctly: Under these particular scoring rules , guessing has a higher expected reward than abstaining. This helps explain why systems optimized for answer accuracy can be reluctant to acknowledge uncertainty. It does not imply every training pipeline explicitly rewards falsehoods. Research by Kalai and colleagues highlights this incentive problem. 4. The model may fail to use available context correctly Hallucinations do not require missing knowledge. Even if a correct document is provided, a model may: Overlook a relevant sentence in a long or distracting prompt. Misinterpret a table, number, or relationship. Blend information from different sources into one incorrect conclusion. Ignore an explicit restriction to answer only from supplied evidence. A RAG system can also fail earlier: retrieval may return a stale, irrelevant, or incomplete passage. The generator might then fill the missing details with an unsupported guess. 5. Generation can amplify earlier mistakes Text is generated sequentially. Once a wrong assumption appears in an answer, subsequent tokens may build upon it, producing more seemingly coherent but incorrect details. Sampling settings can influence which continuation is selected, but changing temperature alone cannot guarantee factual correctness . Even deterministic or greedy decoding can produce a hallucination. Practical example: an internal company assistant Suppose a user asks, "What is the current reimbursement limit for travel?" Data issue: Older training examples contain ₹5,000, but the policy has changed. Retrieval issue: The search tool returns last year's policy instead of the current one. Reasoning issue: The current document distinguishes domestic and international limits, but the model merges them. Guessing issue: No trustworthy policy is available, yet the model confidently invents ₹8,000 rather than acknowledging uncertainty. These produce superficially similar wrong answers, but their fixes are different . How do you diagnose and reduce the cause? Check the source: Did the assistant receive an accurate, current policy or document? Fix retrieval or data freshness if not. Check grounding: If evidence was present, inspect whether the final claim is actually supported by it. Test abstention: Include unanswerable questions and reward correct uncertainty instead of forcing an answer. Validate important claims: Use reliable tools, structured checks, or human review for high impact decisions. Evaluate by failure category: Separate missing source errors from reading errors and fabricated details before selecting a remedy. A low language model perplexity score or fluent writing style does not establish that the output is factual. For the definition and recognizable examples, see "What is hallucination in an LLM?" . Later questions cover hallucination types and evaluation in greater depth. In short: LLM hallucinations arise when uncertain knowledge, imperfect context use, and incentives to produce an answer combine with a generation process that predicts text rather than independently verifying every statement. Research references: Why Language Models Hallucinate, Evaluating large language models for accuracy incentivizes hallucinations, Survey on Hallucination in Large Language Models, and A Survey of RAG Reasoning Systems.

What are the different types of LLM hallucinations?

LLM hallucinations can be classified by whether an answer is wrong about the world or unfaithful to the information and instructions it was given. Researchers use different taxonomies, so there is no single universally agreed list. A useful distinction for evaluating LLM applications is factuality hallucination versus faithfulness hallucination . 1. Factuality hallucinations The response presents information that is false or unverifiable against trustworthy real world evidence . Factual contradiction: The model states an incorrect, checkable fact. Example: “The capital of Australia is Sydney.” The correct capital is Canberra . Factual fabrication: It invents a plausible research paper, court case, statistic, quotation, website, or source that cannot be corroborated. Example: a fabricated journal citation with an invented DOI. How to detect it: Verify material claims against authoritative external sources. A convincing citation is not evidence until you check that it exists and supports the claim. 2. Faithfulness hallucinations The response departs from the user provided document, instructions, or its own reasoning , even when the necessary information is present. Context inconsistency: The document says “Refunds are allowed within 30 days ,” but the assistant replies “You have 60 days .” Instruction inconsistency: The user asks “Extract the three dates appearing in the document,” but the assistant invents a fourth date. Not every instruction following mistake is a hallucination ; it becomes one when unsupported information is generated. Logical inconsistency: A response states “The total is ₹1,200,” then concludes that two equal shares are ₹700 each, contradicting its own arithmetic. How to detect it: Compare each answer claim with the input document and check that reasoning, numerical relationships, and explicitly requested constraints agree. Intrinsic vs. extrinsic hallucinations In source grounded text generation, an influential earlier distinction is: Intrinsic: The generated answer contradicts the provided source . Extrinsic: The answer adds information that cannot be verified from the provided source . Example: A report states “Revenue increased by 10% .” “Revenue decreased by 10%” is an intrinsic contradiction . “Revenue increased by 10% because of a new acquisition” is extrinsic if no acquisition appears in the report. Critical nuance: Extrinsic content is not necessarily factually false . The acquisition could have occurred in reality; it is simply unsupported by that source . If the task requires answers strictly grounded in the report, including it is an ungrounded/faithfulness error . Hallucination research does not always use these labels identically, particularly for open ended LLMs. Other practical labels These describe where the error appears rather than forming completely separate, mutually exclusive categories: Temporal hallucination: Using an obsolete date or treating an outdated policy as current. Citation hallucination: Inventing a publication or attaching a real citation that does not support the claim. Numerical hallucination: Fabricating a figure or reporting arithmetic inconsistent with the data. Tool/API hallucination: Inventing a function name, unsupported API parameter, or a tool result that was never returned. Multimodal hallucination: Describing an object, text, or detail that is not present in an image or other provided modality. A single answer can fall into multiple labels . For example, an invented citation is both a fabrication and a citation hallucination. Beginner friendly evaluation example Suppose a document grounded support assistant answers 20 customer questions. Manual review finds: 2 answers that contradict the policy. 3 answers containing unsupported extra claims. 1 answer with both a contradiction and an unsupported claim. If the categories above count answers , there are 4 distinct affected answers : 2 + 3 − 1 = 4. Do not simply add category counts when errors overlap; that would count the same answer twice. For claim level metrics or multiple labels per answer, use the correct denominator. Why does this classification matter? It helps you choose the right fix: False real world claim: Improve evidence access, freshness, and external verification. Wrong reading of supplied evidence: Improve chunk relevance, prompt grounding, or document understanding. Invented citation/API: Validate references and tool schemas before presenting them. Internal arithmetic contradiction: Use deterministic calculations or consistency checks. In short: First ask “Is the claim true?” (factuality) and “Is it supported by the supplied input?” (faithfulness) . Those are different questions, and both matter in reliable LLM applications. For the basic definition, see “What is hallucination in an LLM?” ; for root causes, see “Why do Large Language Models hallucinate?” . The later evaluation question covers methods for measuring these errors systematically. Research references: A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions, HalluLens: LLM Hallucination Benchmark, and Large Language Models Hallucination: A Comprehensive Survey.