Voyager: An Open-Ended Embodied Agent with Large Language Models
Key takeaway
Foundational work on long-term learning and skill library.
Updated on Oct 3, 2026
01
Abstract
- Voyager is presented as an LLM-powered embodied lifelong-learning agent for Minecraft.
- It continuously explores an open-ended environment without human intervention.
- Its objective is not a single fixed task but continual acquisition of diverse skills and discoveries.
- Voyager combines automatic curriculum generation, an executable skill library, and iterative prompting.
- GPT-4 is used through black-box API queries rather than parameter fine-tuning.
- Learned skills are temporally extended, interpretable, and compositional.
- The skill library lets later behaviors reuse earlier code instead of relearning from scratch.
- The paper reports substantially stronger exploration and technology-tree progression than prior methods.
02
Introduction
- Open-ended embodied learning requires an agent to choose what to do next while operating in a changing environment.
- Fixed task lists do not capture lifelong exploration or continual skill acquisition.
- Reinforcement-learning agents can be expensive to train and may suffer catastrophic forgetting when objectives change.
- LLMs provide broad world knowledge and code-generation ability but need an environment-grounded interaction loop.
- Minecraft provides a large action space, persistent state, long-horizon dependencies, and a meaningful technology tree.
- Voyager treats executable code as a reusable representation of learned behavior.
- The paper combines exploration, memory, feedback, and self-improvement rather than optimizing a single reward function.
03
Problem
- The central problem is to build an agent that can autonomously explore Minecraft and acquire increasingly complex skills.
- The agent must select useful goals without a human specifying every task.
- It must translate goals into executable actions.
- Generated programs can fail because of syntax, runtime, or logical errors.
- Environment feedback must therefore be fed back into subsequent program generation.
- Previously learned skills should remain available as reusable primitives.
- The agent should transfer acquired skills to a new Minecraft world without retraining.
- Evaluation must measure open-ended exploration and progression rather than only success on a fixed task list.
04
Methodology
- Voyager decomposes lifelong learning into three major components: automatic curriculum, skill library, and iterative prompting.
- The automatic curriculum proposes increasingly useful exploration objectives based on the current state.
- The skill library stores successful executable programs for later retrieval.
- Iterative prompting asks GPT-4 to generate or revise code based on environment feedback.
- Execution errors are returned to the model as explicit feedback.
- Self-verification is used to assess whether generated programs achieved the intended objective.
- Retrieved skills can be composed with newly generated code to solve more complex tasks.
- The loop repeats continuously, allowing capabilities to accumulate over time.
How Voyager turns exploration into reusable skills
Read the diagram as a repeating loop. On the left, the automatic curriculum selects a new task based on exploration progress, such as making a crafting table or fighting a zombie. In the center, iterative prompting produces executable code and retrieves useful programs from the skill library on the right. For the combat task, the program can reuse sword-crafting and shield-crafting skills rather than rebuilding those behaviors. The code runs in Minecraft, which returns observations and execution errors. Self-verification checks whether the task was achieved: failure sends feedback back to refine the program, while success adds a reusable skill to the library. Progress also feeds back into the curriculum to guide the next task. The arrows connect task selection, skill retrieval, execution, checking, and revision into a system that accumulates more complex behaviors over time.
Source: Guanzhi Wang et al., Voyager: An Open-Ended Embodied Agent with Large Language Models, Figure 2 (arXiv:2305.16291).
05
Architecture
- The automatic curriculum is designed to maximize exploration rather than simply finish a predefined task.
- It proposes goals conditioned on the agent's current knowledge and progress.
- The skill library is an ever-growing repository of executable behaviors.
- Each skill can serve as a temporally extended primitive for later plans.
- The iterative prompting mechanism includes code generation, execution, feedback, and revision.
- Environment feedback includes the observed outcome of executing a generated program.
- Execution errors are incorporated into the next prompt to enable debugging.
- Self-verification checks whether the generated behavior satisfies the target goal.
- The three components form a closed loop: explore, acquire skill, retrieve skill, compose, and explore again.
06
Dataset
- The experiments use Minecraft as the embodied environment.
- The evaluation is based on open-ended exploration and the Minecraft technology tree.
- Item collection is used as a measure of breadth of acquired capabilities.
- Travel distance measures the agent's ability to explore the environment.
- Technology-tree milestones measure progress toward increasingly advanced capabilities.
- A new-world experiment tests whether the accumulated skill library transfers to an unseen Minecraft world.
- The paper evaluates both exploration during lifelong learning and downstream use of learned skills.
- The environment provides a persistent world in which skills can depend on previously acquired resources and tools.
07
Training
- Voyager uses GPT-4 through black-box queries.
- No parameter fine-tuning is performed on GPT-4 during the lifelong-learning process.
- Prompts contain the current goal, relevant environment information, generated code, execution feedback, and retrieved skills when appropriate.
- Failed programs are revised using explicit error messages and environmental outcomes.
- Successful behaviors are converted into reusable skills and stored.
- The persistent skill library functions as an external memory of executable behavior.
- This design shifts learning from weight updates to accumulated programs and contextual prompting.
08
Experiments
- The main experiments measure how quickly and broadly Voyager explores Minecraft.
- Exploration is compared against prior LLM-agent and embodied-agent approaches.
- The paper reports unique-item acquisition, travel distance, and technology-tree milestone speed.
- A transfer experiment evaluates a Voyager agent in a new Minecraft world using its existing skill library.
- The study also analyzes whether compositional skills enable increasingly complex behaviors.
- The authors examine the contribution of the curriculum, skill library, and iterative prompting through controlled comparisons.
- Qualitative examples illustrate behaviors that emerge from combining previously learned skills.
09
Baselines
- Baselines include prior LLM-based Minecraft agents and exploration-oriented embodied approaches.
- The comparison includes AutoGPT and ReAct-style approaches adapted to Minecraft.
- Other baselines include Voyager-component variants that remove or replace major mechanisms.
- The key comparison is open-ended autonomous exploration rather than only task completion.
- New-world transfer provides an additional comparison because a system that memorizes one world should not automatically generalize.
- Metrics are reported as relative exploration and progression improvements against the strongest prior baseline.
10
Results
- Voyager obtains 3.3× more unique items than the prior state of the art.
- Voyager travels 2.3× longer distances than the prior state of the art.
- Voyager unlocks key technology-tree milestones up to 15.3× faster than prior state of the art.
- The learned skill library can be reused in a new Minecraft world.
- Voyager solves novel tasks in the new world from scratch using previously acquired skills.
- Compounding skill acquisition allows later behaviors to be built from earlier primitives.
- The results support the claim that code-based skills provide a persistent form of lifelong memory.
- The paper reports strong in-context lifelong-learning behavior without fine-tuning GPT-4.
11
Conclusion
- Voyager demonstrates a practical architecture for open-ended embodied lifelong learning with an LLM.
- Automatic curriculum lets the agent decide what to explore next.
- The skill library converts successful behaviors into reusable executable memory.
- Iterative prompting turns execution feedback and errors into opportunities for program repair.
- Compositional skills allow capabilities to compound over time.
- The new-world experiment demonstrates transfer of accumulated skills beyond the original environment state.
- The paper's broader contribution is showing that LLM agents can learn through accumulated programs without changing model weights.
12
References
- Primary paper: Guanzhi Wang et al., “Voyager: An Open-Ended Embodied Agent with Large Language Models.”
- arXiv:2305.16291.
- Authors: Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar.
- The project website is https://voyager.minedojo.org/.
- The paper reports experiments in Minecraft using GPT-4 black-box queries.
- Primary source: https://arxiv.org/abs/2305.16291.
Source trail
Primary papers, implementation pages, and external source records used to ground this explainer.
Continue reading
Related research
GitSwarm
GitSwarm turns inference-time work into a persistent, branchable body of reusable contributions that later agents can build on.
AI AgentsLongHarness
LongHarness helps agents reason over 100M-token document collections by keeping evidence and conclusions as explicit, revisable states that guide one another.
AI AgentsGenerative Agents
An agent architecture combining memory, retrieval, reflection, and planning to simulate believable long-horizon behavior.