Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection

PAPER-DERIVED FACTS: Self-RAG trains a single language model to retrieve external passages when needed, generate text, and reflect on both retrieved evidence and its own output. Reflection tokens make these behaviors controllable during inference.

PAPER-DERIVED FACTS: Fixed top-k retrieval can add irrelevant context when retrieval is unnecessary and can still miss useful evidence when the fixed set is poor. Self-RAG makes retrieval and critique adaptive to the current generation task.

PAPER-DERIVED FACTS: The system must decide whether retrieval is necessary, whether retrieved passages are relevant, whether a generated statement is supported, and which candidate output is preferable. These decisions should be integrated with generation rather than bolted on afterward.

PAPER-DERIVED FACTS: Self-RAG builds on retrieval-augmented generation, instruction tuning, and self-reflection. Its main contribution is to train reflection tokens that represent retrieval and critique decisions.

PAPER-DERIVED FACTS: The model generates special reflection tokens that control retrieval and evaluate relevance, support, and usefulness. It can retrieve on demand, generate multiple candidate segments, critique them, and continue generation using the selected evidence.

PAPER-DERIVED FACTS: Self-RAG uses one language model with special reflection tokens rather than a separate controller. The tokens encode actions such as retrieval and judgments about relevance and support, enabling a single model to coordinate the loop.

PAPER-DERIVED FACTS: Training data is constructed for retrieval, generation, and critique behavior, while evaluation spans open-domain QA, reasoning, fact verification, and long-form generation. Reported model sizes include 7B and 13B parameters.

PAPER-DERIVED FACTS: The model is trained to predict both ordinary language tokens and reflection tokens. Retrieval and critique supervision are incorporated so the model learns when external evidence improves generation.

PAPER-DERIVED FACTS: The evaluation compares Self-RAG with language models and retrieval-augmented baselines on factuality, citation accuracy, QA, reasoning, and long-form generation.

PAPER-DERIVED FACTS: Comparisons include ChatGPT, Llama-2-Chat retrieval systems, and other state-of-the-art LLM/RAG approaches. The key baseline distinction is fixed retrieval versus adaptive retrieval and reflection.

PAPER-DERIVED FACTS: The authors report improvements over the compared LLM and RAG baselines across open-domain QA, reasoning, fact verification, factuality, and citation accuracy. The model also improves long-form generation quality by filtering unsupported content.

PAPER-DERIVED FACTS: The study examines the contribution of retrieval, relevance reflection, support reflection, and critique-based selection. Removing reflection components reduces the model's ability to control retrieval and output quality.

PAPER-DERIVED FACTS: Self-RAG can still make mistakes when retrieval is incomplete or when the reflection model incorrectly judges evidence support. Critique tokens improve controllability but do not make the system infallible.

PAPER-DERIVED FACTS: Self-RAG integrates retrieval, generation, and critique into a single trainable model. It provides a general framework for adaptive RAG where retrieval is invoked only when the model expects external evidence to help.

PAPER-DERIVED FACTS: Primary source: Asai, A. et al. (2024), Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection. ICLR 2024. arXiv:2310.11511.