Research

AgentBench: Evaluating LLMs as Agents

Xiao Liu12 min readResearch
FOUNDATIONALAI AgentsAdvanced

Key takeaway

Foundational work on general agent evaluation.

Updated on Oct 3, 2026

01

Abstract

  • AgentBench is a multi-dimensional benchmark for evaluating LLMs as autonomous agents.
  • The benchmark contains 8 distinct interactive environments.
  • Evaluation uses multi-turn, open-ended generation rather than static question answering alone.
  • The benchmark evaluates reasoning, decision making, instruction following, and interaction with environments.
  • The paper evaluates 27 API-based and open-source LLMs.
  • Results show a substantial performance gap between leading commercial models and open-source models.
  • The benchmark is released with datasets, environments, and an integrated evaluation package.

02

Introduction

  • Traditional LLM benchmarks mostly evaluate isolated language tasks rather than sustained interaction.
  • Agentic systems must perceive environment state, decide actions, execute them, and adapt to feedback.
  • Long-horizon reasoning and decision making introduce failure modes not captured by single-turn QA.
  • AgentBench therefore evaluates agents in diverse environments instead of relying on one task family.
  • The benchmark is intended to measure general agent capability rather than one particular agent architecture.
  • The paper identifies poor long-term reasoning, decision making, and instruction following as major obstacles to usable agents.
  • It also studies whether code-oriented training and high-quality multi-turn alignment data improve agent behavior.

03

Problem

  • The benchmark asks whether current LLMs can reliably act as agents across heterogeneous interactive environments.
  • A useful evaluation must separate language generation quality from action and decision quality.
  • Agents need to maintain state over multiple turns.
  • They must interpret environment feedback and modify subsequent actions.
  • Different environments impose different constraints, action spaces, and success criteria.
  • A benchmark therefore needs common evaluation infrastructure while preserving environment-specific interfaces.
  • The study seeks to compare both proprietary API models and open-source models under consistent conditions.

04

Methodology

  • AgentBench defines a standardized agent-environment interaction loop.
  • An LLM receives a task and current environment observation.
  • The model generates an action or action-formatted response.
  • The environment executes the action and returns the resulting state or feedback.
  • The process repeats until task completion, failure, or a maximum interaction budget.
  • Environment-specific wrappers normalize the interaction interface for different tasks.
  • Evaluation metrics are defined by each environment's task success or reward signal.
  • The framework supports both API-based models and locally hosted open-source models.
  • The integrated evaluator records interactions so failure patterns can be analyzed across models and environments.
Figure 1

05

Architecture

  • The benchmark contains 8 environments: Operating System, Database, Knowledge Graph, Digital Card Game, Web Shopping, Web Browsing, Household, and Lateral Thinking.
  • Operating System tasks require command-line interaction and file/system operations.
  • Database tasks require interacting with databases through structured queries or actions.
  • Knowledge Graph tasks test multi-step information retrieval and reasoning.
  • Digital Card Game tests sequential decision making under game rules.
  • Web Shopping tests goal-directed interaction with an online shopping environment.
  • Web Browsing tests navigation and information-seeking behavior.
  • Household tasks test embodied-style planning in a simulated environment.
  • Lateral Thinking tests reasoning through interactive question-and-answer behavior.

06

Dataset

  • AgentBench is an evaluation benchmark rather than a model-training method.
  • The main study evaluates pretrained API and open-source LLMs directly.
  • The paper examines whether model training characteristics correlate with agent performance.
  • Additional analysis tests the effect of code-oriented training and multi-turn alignment data.
  • Code training can improve performance in environments requiring executable actions.
  • High-quality multi-turn alignment data can improve an LLM's ability to follow interaction protocols.
  • The benchmark therefore separates general language capability from agent-specific interaction competence.

07

Training

  • The paper evaluates 27 LLMs spanning commercial API models and open-source models.
  • The evaluated systems include GPT-4, ChatGPT/GPT-3.5-era models, and multiple open-source families.
  • All models are evaluated through the benchmark's standardized environment interfaces.
  • Performance is measured across all eight environments and also by environment-specific scores.
  • The study examines correlations between conventional NLP/LLM capability and agent performance.
  • Failure analysis groups errors by reasoning, decision making, instruction following, and environment interaction.
  • The paper also studies how training data characteristics relate to agent performance.

08

Experiments

  • The strongest commercial models substantially outperform open-source competitors across AgentBench.
  • GPT-4 is among the strongest overall agents in the reported benchmark.
  • Performance varies strongly by environment, showing that agent ability is not captured by one aggregate language score.
  • Long-horizon tasks expose failures that are not obvious on conventional single-turn benchmarks.
  • Code-trained models tend to perform better on environments requiring programmatic actions.
  • Models with stronger multi-turn alignment are better at following interaction protocols.
  • The paper concludes that current LLMs still struggle with reliable long-term planning and decision making.
  • AgentBench provides evidence that general LLM strength does not automatically translate into robust agent behavior.

09

Baselines

  • Baselines include the evaluated commercial API models and open-source LLMs under a common interaction protocol.
  • Conventional language-model capability provides an implicit baseline for comparing agent performance.
  • Different model families are compared rather than treating one prompting strategy as the benchmark baseline.
  • The benchmark also analyzes environment-specific baselines where the environment supplies its own scoring mechanism.
  • The key comparison is cross-model performance under identical task interfaces.
  • This design reduces confounding from different external agent frameworks.

10

Results

  • AgentBench demonstrates a substantial gap between top commercial models and open-source models.
  • GPT-4 shows strong performance across many of the eight environments.
  • Performance is uneven: a model can perform well in one environment while failing in another.
  • Long-horizon reasoning remains a major limitation even for strong commercial LLMs.
  • Instruction-following failures can cause an otherwise capable model to perform invalid actions.
  • Code-oriented training improves performance in action spaces that benefit from programmatic reasoning.
  • Multi-turn alignment data improves the ability to sustain useful interaction over multiple steps.
  • The results motivate agent-specific evaluation rather than relying only on standard LLM benchmarks.

11

Conclusion

  • AgentBench establishes a common evaluation framework for LLM agents across eight heterogeneous environments.
  • Its multi-turn interaction design captures reasoning and decision making under environmental feedback.
  • The benchmark demonstrates large differences among models that may look similar on conventional language evaluations.
  • Commercial frontier models are substantially stronger, but even they show long-horizon weaknesses.
  • Code training and multi-turn alignment are identified as useful directions for improving agent capability.
  • The paper's central lesson is that agent evaluation must measure acting, not only answering.
  • The released environments and evaluation package provide a foundation for subsequent agent benchmarks.

12

References

  • Primary paper: Xiao Liu et al., “AgentBench: Evaluating LLMs as Agents,” arXiv:2308.03688.
  • The paper was published at ICLR 2024.
  • Authors include Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, Shudan Zhang, Xiang Deng, Aohan Zeng, Zhengxiao Du, Chenhui Zhang, Sheng Shen, Tianjun Zhang, Yu Su, Huan Sun, Minlie Huang, Yuxiao Dong, and Jie Tang.
  • The benchmark contains 8 environments and evaluates 27 API-based and open-source LLMs.
  • Project repository: https://github.com/THUDM/AgentBench.
  • Primary source: https://arxiv.org/abs/2308.03688.

Source trail

Primary papers, implementation pages, and external source records used to ground this explainer.

Continue reading

Related research

Browse all research