Research

ReAct: Synergizing Reasoning and Acting in Language Models

ReAct combines reasoning and acting in an iterative Reason → Act → Observe loop.

Shunyu Yao12 min readPrinceton University / Google Research
FOUNDATIONALAI AgentsAdvancedAI Agents

Key takeaway

A foundational agent pattern that interleaves reasoning with environment actions and observations.

Updated on Oct 3, 2026

01

Abstract

  • ReAct interleaves language-based reasoning traces with task-specific actions rather than treating reasoning and acting as separate capabilities.
  • Reasoning helps create, track, and revise action plans; actions retrieve information from external environments that feeds the next reasoning step.
  • The paper evaluates this paradigm on HotpotQA, FEVER, ALFWorld, and WebShop and reports gains in both knowledge-intensive reasoning and interactive decision making.

02

Introduction

  • The motivation is inspired by human problem solving, where internal verbal reasoning and external actions are interleaved to maintain progress and adapt to uncertainty.
  • Chain-of-thought reasoning is described as a static process that is not grounded in external observations, making factual hallucination and error propagation possible.
  • Action-only approaches can interact with environments but may lack high-level reasoning, working-memory-like state tracking, and systematic subgoal planning.
  • ReAct is presented as a general prompting paradigm rather than a new neural architecture.

03

Problem

  • The central problem is how to map a long interaction context to the next action when successful behavior requires implicit multi-step reasoning.
  • Reasoning-only systems can hallucinate facts because their intermediate conclusions are not checked against the environment.
  • Acting-only systems can repeatedly issue ineffective actions because they do not explicitly reason about what the observations imply for the high-level goal.
  • ReAct addresses both failure modes by adding language thoughts to the action space and feeding observations back into the next reasoning step.

04

Methodology

  • The agent receives an observation, reasons over the current trajectory, selects an action or thought, observes the result, and continues from the expanded context.
  • The authors manually construct few-shot trajectories containing Thought, Action, and Observation steps.
  • For knowledge tasks, thoughts are relatively dense and can decompose questions, extract facts, perform commonsense or arithmetic reasoning, reformulate searches, and synthesize answers.
  • For long-horizon decision tasks, thoughts are intentionally sparse and the model decides when a thought is useful.
  • The main prompting experiments use a frozen PaLM-540B model; the paper also reports GPT-3 experiments in the appendix.
Figure 1

05

Architecture

  • ReAct defines an augmented action space A-hat = A union L, where A contains environment actions and L contains language thoughts.
  • A thought does not produce an environment observation; instead it updates the internal trajectory context for future reasoning or acting.
  • Useful thoughts include decomposing goals, injecting commonsense, extracting important observations, tracking progress, reformulating plans, and handling exceptions.
  • The framework is flexible because the thought space is free-form and the occurrence of thoughts can be dense or sparse depending on the task.

06

Dataset

  • HotpotQA tests multi-hop question answering over two or more Wikipedia passages.
  • FEVER tests whether claims are SUPPORTS, REFUTES, or NOT ENOUGH INFO using Wikipedia evidence.
  • ALFWorld is a text-based household environment with six task types and potentially more than 50 locations in one instance.
  • WebShop contains about 1.18 million real-world products and 12,000 human instructions and requires navigating noisy product information.
  • HotpotQA and FEVER use a question-only setup so the model must rely on internal knowledge or interactively retrieve evidence.

07

Training

  • The primary experiments are few-shot prompting with a frozen PaLM-540B model rather than full model training.
  • HotpotQA and FEVER use 6 and 3 manually composed ReAct trajectories respectively as in-context exemplars.
  • For ALFWorld, three trajectories are annotated per task type and sparse thoughts cover goal decomposition, subgoal tracking, next-subgoal selection, and commonsense.
  • The authors bootstrap fine-tuning using 3,000 correct ReAct-generated trajectories to train PaLM-8B and PaLM-62B.
  • Fine-tuning ReAct is substantially more effective than fine-tuning Standard or CoT in the reported scaling experiments.

08

Experiments

  • The knowledge-intensive experiments compare Standard, CoT, CoT self-consistency, Act-only, ReAct, and two hybrid ReAct/CoT-SC strategies.
  • The Wikipedia API exposes search[entity], lookup[string], and finish[answer] actions.
  • The retrieval interface is deliberately weaker than modern lexical or neural retrievers so that successful retrieval depends on explicit reasoning.
  • ALFWorld evaluates on 134 unseen games; WebShop evaluates on 500 test instructions.
  • The authors also study scaling, fine-tuning, prompt robustness, and qualitative trajectories.

09

Baselines

  • Standard prompting removes thoughts, actions, and observations from the ReAct trajectories.
  • CoT removes actions and observations, leaving reasoning-only traces.
  • CoT-SC samples 21 reasoning trajectories at temperature 0.7 and takes the majority answer.
  • Act-only removes thoughts and retains environment actions, providing a controlled test of whether reasoning adds value.
  • The paper also compares against supervised approaches and interactive-learning baselines such as BUTLER and IL/IL+RL.

10

Results

  • On PaLM-540B, ReAct scores 27.4 EM on HotpotQA and 60.9 accuracy on FEVER.
  • CoT-SC followed by ReAct reaches 34.2 HotpotQA EM and 64.6 FEVER accuracy; ReAct followed by CoT-SC reaches 35.1 and 62.0.
  • On ALFWorld, the best ReAct trial reaches 71% success versus 45% for Act and 37% for BUTLER.
  • On WebShop, ReAct reaches 66.6 average score and 40.0% success, versus 62.3/30.1 for Act.
  • The paper reports absolute improvements of 34% and 10% over relevant prior methods on ALFWorld and WebShop in the headline results.

11

Conclusion

  • ReAct demonstrates that reasoning and acting can form a closed-loop control pattern for LLM agents across knowledge retrieval and interactive environments.
  • The method improves interpretability because humans can inspect the reasoning/action trajectory and distinguish internal knowledge from external observations.
  • Fine-tuning results suggest the approach can become substantially stronger when the model is trained on high-quality trajectories rather than relying only on few-shot prompting.
  • The authors identify multi-task training, more human-written annotations, and reinforcement learning as promising directions.

12

References

Primary source: Yao, Shunyu; Zhao, Jeffrey; Yu, Dian; Du, Nan; Shafran, Izhak; Narasimhan, Karthik; Cao, Yuan. “ReAct: Synergizing Reasoning and Acting in Language Models.” ICLR 2023, arXiv:2210.03629.

  • The paper includes a reproducibility statement with prompts in Appendix C and additional GPT-3 experiments/code.
  • The paper explicitly discusses safety risks when LLMs are connected to external action spaces, including private-information access and harmful actions.
  • The benchmark environments were constrained to reduce risk: Wikipedia/WebShop interactions had no private information and purchasing/editing actions were disabled.

Source trail

Primary papers, implementation pages, and external source records used to ground this explainer.

Continue reading

Related research

Browse all research