Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions

PAPER-DERIVED FACTS: IRCoT interleaves retrieval operations with chain-of-thought reasoning for knowledge-intensive multi-step questions. Reasoning steps guide retrieval, and retrieved evidence in turn improves the next reasoning step.

PAPER-DERIVED FACTS: A single retrieval query often cannot identify all evidence required for a multi-hop answer. IRCoT treats reasoning as an iterative process in which each derived fact changes the next information need.

PAPER-DERIVED FACTS: The key challenge is dependency between reasoning and retrieval: what should be retrieved depends on what has already been inferred. A static top-k retrieval set can therefore omit later evidence.

PAPER-DERIVED FACTS: Chain-of-thought prompting improves decomposition of multi-step questions, while retrieval supplies external facts. IRCoT combines these two mechanisms without requiring additional model training.

PAPER-DERIVED FACTS: The model generates a reasoning sentence, uses that sentence or its relevant content as a retrieval query, adds retrieved passages to the context, and continues reasoning. The loop terminates when the answer is derived.

PAPER-DERIVED FACTS: IRCoT is a simple iterative controller around an LLM, a retriever, and a reasoning prompt. There is no specialized learned retriever required by the method; existing retrievers and prompting models can be used.

PAPER-DERIVED FACTS: Experiments use HotpotQA, 2WikiMultihopQA, MuSiQue, and IIRC, all of which require combining information across multiple pieces of evidence.

PAPER-DERIVED FACTS: The approach is prompting-based and does not require additional training for the reasoning loop. Experiments use GPT-3 and smaller models such as Flan-T5-large to test transfer across model scales.

PAPER-DERIVED FACTS: The study measures both retrieval quality and final QA performance, including out-of-distribution settings. It compares one-shot retrieval against retrieval guided by generated reasoning steps.

PAPER-DERIVED FACTS: Baselines include standard retrieve-then-read pipelines and chain-of-thought without iterative retrieval. The comparisons isolate the value of allowing reasoning and retrieval to influence each other.

PAPER-DERIVED FACTS: IRCoT reports improvements of up to 21 points in retrieval and up to 15 points in downstream QA on the studied datasets. The method also reduces factual errors in generated reasoning compared with unsupported chain-of-thought.

PAPER-DERIVED FACTS: The experiments examine the role of iterative retrieval and reasoning guidance. Removing either the retrieval loop or the reasoning signal weakens the ability to discover evidence needed for later hops.

PAPER-DERIVED FACTS: IRCoT can propagate an early reasoning error into later retrieval queries, producing irrelevant evidence. Retrieval quality and stopping criteria also affect the final answer.

PAPER-DERIVED FACTS: IRCoT reframes multi-hop QA as a coupled retrieval-and-reasoning process. It provides a lightweight route to iterative RAG without changing the underlying language model.

PAPER-DERIVED FACTS: Primary source: Trivedi, H., Balasubramanian, N., Khot, T., & Sabharwal, A. (2023), Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions. ACL 2023. arXiv:2212.10509.