ReAct: Synergizing Reasoning and Acting in Language Models
ReAct combines reasoning and acting in an iterative Reason → Act → Observe loop.
Key takeaway
A foundational agent pattern that interleaves reasoning with environment actions and observations.
Updated on Oct 3, 2026
01
Abstract
- ReAct interleaves language-based reasoning traces with task-specific actions rather than treating reasoning and acting as separate capabilities.
- Reasoning helps create, track, and revise action plans; actions retrieve information from external environments that feeds the next reasoning step.
- The paper evaluates this paradigm on HotpotQA, FEVER, ALFWorld, and WebShop and reports gains in both knowledge-intensive reasoning and interactive decision making.
02
Introduction
- The motivation is inspired by human problem solving, where internal verbal reasoning and external actions are interleaved to maintain progress and adapt to uncertainty.
- Chain-of-thought reasoning is described as a static process that is not grounded in external observations, making factual hallucination and error propagation possible.
- Action-only approaches can interact with environments but may lack high-level reasoning, working-memory-like state tracking, and systematic subgoal planning.
- ReAct is presented as a general prompting paradigm rather than a new neural architecture.
03
Problem
- The central problem is how to map a long interaction context to the next action when successful behavior requires implicit multi-step reasoning.
- Reasoning-only systems can hallucinate facts because their intermediate conclusions are not checked against the environment.
- Acting-only systems can repeatedly issue ineffective actions because they do not explicitly reason about what the observations imply for the high-level goal.
- ReAct addresses both failure modes by adding language thoughts to the action space and feeding observations back into the next reasoning step.
04
Methodology
- The agent receives an observation, reasons over the current trajectory, selects an action or thought, observes the result, and continues from the expanded context.
- The authors manually construct few-shot trajectories containing Thought, Action, and Observation steps.
- For knowledge tasks, thoughts are relatively dense and can decompose questions, extract facts, perform commonsense or arithmetic reasoning, reformulate searches, and synthesize answers.
- For long-horizon decision tasks, thoughts are intentionally sparse and the model decides when a thought is useful.
- The main prompting experiments use a frozen PaLM-540B model; the paper also reports GPT-3 experiments in the appendix.
05
Architecture
- ReAct defines an augmented action space A-hat = A union L, where A contains environment actions and L contains language thoughts.
- A thought does not produce an environment observation; instead it updates the internal trajectory context for future reasoning or acting.
- Useful thoughts include decomposing goals, injecting commonsense, extracting important observations, tracking progress, reformulating plans, and handling exceptions.
- The framework is flexible because the thought space is free-form and the occurrence of thoughts can be dense or sparse depending on the task.
06
Dataset
- HotpotQA tests multi-hop question answering over two or more Wikipedia passages.
- FEVER tests whether claims are SUPPORTS, REFUTES, or NOT ENOUGH INFO using Wikipedia evidence.
- ALFWorld is a text-based household environment with six task types and potentially more than 50 locations in one instance.
- WebShop contains about 1.18 million real-world products and 12,000 human instructions and requires navigating noisy product information.
- HotpotQA and FEVER use a question-only setup so the model must rely on internal knowledge or interactively retrieve evidence.
07
Training
- The primary experiments are few-shot prompting with a frozen PaLM-540B model rather than full model training.
- HotpotQA and FEVER use 6 and 3 manually composed ReAct trajectories respectively as in-context exemplars.
- For ALFWorld, three trajectories are annotated per task type and sparse thoughts cover goal decomposition, subgoal tracking, next-subgoal selection, and commonsense.
- The authors bootstrap fine-tuning using 3,000 correct ReAct-generated trajectories to train PaLM-8B and PaLM-62B.
- Fine-tuning ReAct is substantially more effective than fine-tuning Standard or CoT in the reported scaling experiments.
08
Experiments
- The knowledge-intensive experiments compare Standard, CoT, CoT self-consistency, Act-only, ReAct, and two hybrid ReAct/CoT-SC strategies.
- The Wikipedia API exposes search[entity], lookup[string], and finish[answer] actions.
- The retrieval interface is deliberately weaker than modern lexical or neural retrievers so that successful retrieval depends on explicit reasoning.
- ALFWorld evaluates on 134 unseen games; WebShop evaluates on 500 test instructions.
- The authors also study scaling, fine-tuning, prompt robustness, and qualitative trajectories.
09
Baselines
- Standard prompting removes thoughts, actions, and observations from the ReAct trajectories.
- CoT removes actions and observations, leaving reasoning-only traces.
- CoT-SC samples 21 reasoning trajectories at temperature 0.7 and takes the majority answer.
- Act-only removes thoughts and retains environment actions, providing a controlled test of whether reasoning adds value.
- The paper also compares against supervised approaches and interactive-learning baselines such as BUTLER and IL/IL+RL.
10
Results
- On PaLM-540B, ReAct scores 27.4 EM on HotpotQA and 60.9 accuracy on FEVER.
- CoT-SC followed by ReAct reaches 34.2 HotpotQA EM and 64.6 FEVER accuracy; ReAct followed by CoT-SC reaches 35.1 and 62.0.
- On ALFWorld, the best ReAct trial reaches 71% success versus 45% for Act and 37% for BUTLER.
- On WebShop, ReAct reaches 66.6 average score and 40.0% success, versus 62.3/30.1 for Act.
- The paper reports absolute improvements of 34% and 10% over relevant prior methods on ALFWorld and WebShop in the headline results.
11
Conclusion
- ReAct demonstrates that reasoning and acting can form a closed-loop control pattern for LLM agents across knowledge retrieval and interactive environments.
- The method improves interpretability because humans can inspect the reasoning/action trajectory and distinguish internal knowledge from external observations.
- Fine-tuning results suggest the approach can become substantially stronger when the model is trained on high-quality trajectories rather than relying only on few-shot prompting.
- The authors identify multi-task training, more human-written annotations, and reinforcement learning as promising directions.
12
References
Primary source: Yao, Shunyu; Zhao, Jeffrey; Yu, Dian; Du, Nan; Shafran, Izhak; Narasimhan, Karthik; Cao, Yuan. “ReAct: Synergizing Reasoning and Acting in Language Models.” ICLR 2023, arXiv:2210.03629.
- The paper includes a reproducibility statement with prompts in Appendix C and additional GPT-3 experiments/code.
- The paper explicitly discusses safety risks when LLMs are connected to external action spaces, including private-information access and harmful actions.
- The benchmark environments were constrained to reduce risk: Wikipedia/WebShop interactions had no private information and purchasing/editing actions were disabled.
Source trail
Primary papers, implementation pages, and external source records used to ground this explainer.
Continue reading
Related research
GitSwarm
GitSwarm turns inference-time work into a persistent, branchable body of reusable contributions that later agents can build on.
AI AgentsLongHarness
LongHarness helps agents reason over 100M-token document collections by keeping evidence and conclusions as explicit, revisable states that guide one another.
AI AgentsGenerative Agents
An agent architecture combining memory, retrieval, reflection, and planning to simulate believable long-horizon behavior.