Research

Voyager: An Open-Ended Embodied Agent with Large Language Models

Guanzhi Wang12 min readResearch
FOUNDATIONALAI AgentsAdvanced

Key takeaway

Foundational work on long-term learning and skill library.

Updated on Oct 3, 2026

01

Abstract

  • Voyager is presented as an LLM-powered embodied lifelong-learning agent for Minecraft.
  • It continuously explores an open-ended environment without human intervention.
  • Its objective is not a single fixed task but continual acquisition of diverse skills and discoveries.
  • Voyager combines automatic curriculum generation, an executable skill library, and iterative prompting.
  • GPT-4 is used through black-box API queries rather than parameter fine-tuning.
  • Learned skills are temporally extended, interpretable, and compositional.
  • The skill library lets later behaviors reuse earlier code instead of relearning from scratch.
  • The paper reports substantially stronger exploration and technology-tree progression than prior methods.

02

Introduction

  • Open-ended embodied learning requires an agent to choose what to do next while operating in a changing environment.
  • Fixed task lists do not capture lifelong exploration or continual skill acquisition.
  • Reinforcement-learning agents can be expensive to train and may suffer catastrophic forgetting when objectives change.
  • LLMs provide broad world knowledge and code-generation ability but need an environment-grounded interaction loop.
  • Minecraft provides a large action space, persistent state, long-horizon dependencies, and a meaningful technology tree.
  • Voyager treats executable code as a reusable representation of learned behavior.
  • The paper combines exploration, memory, feedback, and self-improvement rather than optimizing a single reward function.

03

Problem

  • The central problem is to build an agent that can autonomously explore Minecraft and acquire increasingly complex skills.
  • The agent must select useful goals without a human specifying every task.
  • It must translate goals into executable actions.
  • Generated programs can fail because of syntax, runtime, or logical errors.
  • Environment feedback must therefore be fed back into subsequent program generation.
  • Previously learned skills should remain available as reusable primitives.
  • The agent should transfer acquired skills to a new Minecraft world without retraining.
  • Evaluation must measure open-ended exploration and progression rather than only success on a fixed task list.

04

Methodology

  • Voyager decomposes lifelong learning into three major components: automatic curriculum, skill library, and iterative prompting.
  • The automatic curriculum proposes increasingly useful exploration objectives based on the current state.
  • The skill library stores successful executable programs for later retrieval.
  • Iterative prompting asks GPT-4 to generate or revise code based on environment feedback.
  • Execution errors are returned to the model as explicit feedback.
  • Self-verification is used to assess whether generated programs achieved the intended objective.
  • Retrieved skills can be composed with newly generated code to solve more complex tasks.
  • The loop repeats continuously, allowing capabilities to accumulate over time.
Figure 1

How Voyager turns exploration into reusable skills

Read the diagram as a repeating loop. On the left, the automatic curriculum selects a new task based on exploration progress, such as making a crafting table or fighting a zombie. In the center, iterative prompting produces executable code and retrieves useful programs from the skill library on the right. For the combat task, the program can reuse sword-crafting and shield-crafting skills rather than rebuilding those behaviors. The code runs in Minecraft, which returns observations and execution errors. Self-verification checks whether the task was achieved: failure sends feedback back to refine the program, while success adds a reusable skill to the library. Progress also feeds back into the curriculum to guide the next task. The arrows connect task selection, skill retrieval, execution, checking, and revision into a system that accumulates more complex behaviors over time.

Source: Guanzhi Wang et al., Voyager: An Open-Ended Embodied Agent with Large Language Models, Figure 2 (arXiv:2305.16291).

05

Architecture

  • The automatic curriculum is designed to maximize exploration rather than simply finish a predefined task.
  • It proposes goals conditioned on the agent's current knowledge and progress.
  • The skill library is an ever-growing repository of executable behaviors.
  • Each skill can serve as a temporally extended primitive for later plans.
  • The iterative prompting mechanism includes code generation, execution, feedback, and revision.
  • Environment feedback includes the observed outcome of executing a generated program.
  • Execution errors are incorporated into the next prompt to enable debugging.
  • Self-verification checks whether the generated behavior satisfies the target goal.
  • The three components form a closed loop: explore, acquire skill, retrieve skill, compose, and explore again.

06

Dataset

  • The experiments use Minecraft as the embodied environment.
  • The evaluation is based on open-ended exploration and the Minecraft technology tree.
  • Item collection is used as a measure of breadth of acquired capabilities.
  • Travel distance measures the agent's ability to explore the environment.
  • Technology-tree milestones measure progress toward increasingly advanced capabilities.
  • A new-world experiment tests whether the accumulated skill library transfers to an unseen Minecraft world.
  • The paper evaluates both exploration during lifelong learning and downstream use of learned skills.
  • The environment provides a persistent world in which skills can depend on previously acquired resources and tools.

07

Training

  • Voyager uses GPT-4 through black-box queries.
  • No parameter fine-tuning is performed on GPT-4 during the lifelong-learning process.
  • Prompts contain the current goal, relevant environment information, generated code, execution feedback, and retrieved skills when appropriate.
  • Failed programs are revised using explicit error messages and environmental outcomes.
  • Successful behaviors are converted into reusable skills and stored.
  • The persistent skill library functions as an external memory of executable behavior.
  • This design shifts learning from weight updates to accumulated programs and contextual prompting.

08

Experiments

  • The main experiments measure how quickly and broadly Voyager explores Minecraft.
  • Exploration is compared against prior LLM-agent and embodied-agent approaches.
  • The paper reports unique-item acquisition, travel distance, and technology-tree milestone speed.
  • A transfer experiment evaluates a Voyager agent in a new Minecraft world using its existing skill library.
  • The study also analyzes whether compositional skills enable increasingly complex behaviors.
  • The authors examine the contribution of the curriculum, skill library, and iterative prompting through controlled comparisons.
  • Qualitative examples illustrate behaviors that emerge from combining previously learned skills.

09

Baselines

  • Baselines include prior LLM-based Minecraft agents and exploration-oriented embodied approaches.
  • The comparison includes AutoGPT and ReAct-style approaches adapted to Minecraft.
  • Other baselines include Voyager-component variants that remove or replace major mechanisms.
  • The key comparison is open-ended autonomous exploration rather than only task completion.
  • New-world transfer provides an additional comparison because a system that memorizes one world should not automatically generalize.
  • Metrics are reported as relative exploration and progression improvements against the strongest prior baseline.

10

Results

  • Voyager obtains 3.3× more unique items than the prior state of the art.
  • Voyager travels 2.3× longer distances than the prior state of the art.
  • Voyager unlocks key technology-tree milestones up to 15.3× faster than prior state of the art.
  • The learned skill library can be reused in a new Minecraft world.
  • Voyager solves novel tasks in the new world from scratch using previously acquired skills.
  • Compounding skill acquisition allows later behaviors to be built from earlier primitives.
  • The results support the claim that code-based skills provide a persistent form of lifelong memory.
  • The paper reports strong in-context lifelong-learning behavior without fine-tuning GPT-4.

11

Conclusion

  • Voyager demonstrates a practical architecture for open-ended embodied lifelong learning with an LLM.
  • Automatic curriculum lets the agent decide what to explore next.
  • The skill library converts successful behaviors into reusable executable memory.
  • Iterative prompting turns execution feedback and errors into opportunities for program repair.
  • Compositional skills allow capabilities to compound over time.
  • The new-world experiment demonstrates transfer of accumulated skills beyond the original environment state.
  • The paper's broader contribution is showing that LLM agents can learn through accumulated programs without changing model weights.

12

References

  • Primary paper: Guanzhi Wang et al., “Voyager: An Open-Ended Embodied Agent with Large Language Models.”
  • arXiv:2305.16291.
  • Authors: Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar.
  • The project website is https://voyager.minedojo.org/.
  • The paper reports experiments in Minecraft using GPT-4 black-box queries.
  • Primary source: https://arxiv.org/abs/2305.16291.

Continue reading

Related research

Browse all research