Retrieval-Augmented Generation for Large Language Models: A Survey
A systematic survey organizing RAG architectures, training strategies, applications, evaluation, and open challenges.
Key takeaway
RAG is decomposed into retrieval, augmentation, and generation, with a progression from simple pipelines to modular systems.
Updated on Oct 2, 2026
01
Abstract
Retrieval-Augmented Generation combines parametric language models with external evidence retrieved at inference time. The survey organizes RAG architectures, retrieval methods, augmentation strategies, generation, evaluation, and major failure modes.
02
Introduction
Language models contain broad knowledge but can be limited by stale knowledge, missing domain information, hallucination, and context constraints. RAG introduces an external knowledge access path without requiring all knowledge to be stored in model parameters.
03
Problem
A RAG system must retrieve the right evidence, represent it efficiently, place useful context into the model input, and generate an answer that remains faithful to that evidence. Retrieval errors can propagate directly into generation.
04
Background
RAG connects information retrieval with neural language generation. The design space includes sparse retrieval such as BM25, dense embeddings, hybrid retrieval, reranking, vector indexes, knowledge bases, and different augmentation strategies.
05
Methodology
A typical RAG pipeline indexes documents, retrieves candidates for a query, reranks or filters them, constructs a context, and generates a response. Modern variants add iterative retrieval, query rewriting, corrective retrieval, agentic search, and post-generation verification.
Figure 1: Technology Tree of RAG Research
Original Figure 1 from the RAG survey showing the technology tree of RAG research across pre-training, fine-tuning, and inference stages.
Original Figure 1 from Gao et al., Retrieval-Augmented Generation for Large Language Models: A Survey, arXiv:2312.10997v5.
Formula notes
Mathematical details
Retrieval similarity
Rank candidate documents by similarity between the query embedding and document embedding.
- E
- -embedding function
- d
- -document
- q
- -query
- sim
- -similarity function
06
Architecture
The architecture separates ingestion and indexing from online retrieval and generation. Production systems commonly include document parsing, chunking, embedding, vector or lexical indexes, candidate retrieval, reranking, context construction, generation, citation handling, and evaluation.
07
Dataset
RAG evaluation uses knowledge-intensive QA, open-domain QA, domain-specific corpora, long-document tasks, and retrieval benchmarks. Dataset quality and corpus freshness strongly influence measured performance.
08
Training
RAG can be implemented without retraining the generator. Training can instead target retrievers, rerankers, query reformulators, or the generator itself, depending on the desired optimization point.
09
Experiments
The survey organizes evaluation around retrieval quality, generation quality, factuality, faithfulness, latency, and end-to-end task performance. Common retrieval metrics include Recall@K, Precision@K, MRR, and nDCG, while generation requires task-specific measures.
Evidence highlights
- 01State management
- 02Tool contracts
- 03Observability
- 04Evaluation
- 05Safety
- 06Cost control
10
Baselines
Parametric-only language models, BM25, dense retrieval, hybrid retrieval, and reranked retrieval provide common baselines.
11
Results
RAG can improve knowledge-intensive generation when relevant evidence is retrieved, but retrieval and context quality remain decisive.
12
Ablation
Retrieval, reranking, query rewriting, chunking, context size, citation verification, and generator configuration are key ablation dimensions.
13
Limitations
Poor retrieval, conflicting sources, stale corpora, context overload, and weak citation grounding can reduce reliability.
14
Conclusion
RAG is a system architecture connecting retrieval and generation; production quality depends on the complete pipeline.
15
References
Primary RAG survey and foundational work on retrieval-augmented generation, dense retrieval, BM25, reranking, knowledge-intensive QA, and grounded generation.
Source trail
Primary papers, implementation pages, and external source records used to ground this explainer.
Continue reading