Research

MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework

Sirui Hong12 min readResearch
FOUNDATIONALAI AgentsAdvanced

Key takeaway

Foundational work on agents organized around workflows and sops.

Updated on Oct 3, 2026

01

Abstract

  • MetaGPT is a multi-agent framework that encodes Standardized Operating Procedures (SOPs) into prompts for LLM agents.
  • It targets complex collaborative tasks where naive agent-to-agent chaining can amplify hallucinations and inconsistencies.
  • The framework assigns different software-engineering roles to different agents.
  • A product manager converts requirements into a product requirement document.
  • An architect designs the system and interfaces.
  • An engineer implements the code.
  • A project manager coordinates the workflow and task allocation.
  • The assembly-line organization makes intermediate artifacts explicit.

02

Introduction

  • The paper identifies cascading hallucinations as a key weakness of naive multi-agent conversation.
  • Simply chaining agents can propagate an early incorrect assumption through later stages.
  • Human software teams reduce this risk through structured workflows, role specialization, and intermediate review.
  • MetaGPT transfers this workflow structure into prompts and message-passing protocols.
  • The central idea is meta-programming: use SOPs to constrain how LLM agents collaborate.
  • Software engineering is used as the primary complex-task setting.
  • The framework aims to produce coherent end-to-end software rather than isolated chat responses.

03

Problem

  • The problem is how to make multiple LLM agents cooperate reliably on complex software-development tasks.
  • Agents need role boundaries so that responsibilities do not collapse into unstructured dialogue.
  • Intermediate artifacts must be produced in a predictable format for downstream agents.
  • Later agents need access to validated upstream information rather than raw conversational history alone.
  • The system needs a mechanism for decomposing a broad requirement into implementation tasks.
  • Coordination must remain consistent as requirements, architecture, and code evolve.
  • The paper evaluates whether structured collaboration improves coherence over chat-based multi-agent baselines.

04

Methodology

  • The workflow begins with a natural-language requirement.
  • A product manager agent analyzes requirements and produces a product requirement document.
  • An architect agent uses the requirement document to produce system design information.
  • A project manager decomposes the work into implementation tasks.
  • Engineer agents implement the assigned tasks.
  • Agents use structured intermediate outputs to communicate rather than relying only on conversational turns.
  • SOP prompts define what each role should do and what information it should produce.
  • The process is sequential enough to preserve dependency order while allowing specialized agents to work on subtasks.
Figure 1

From a requirement to a tested software project

The diagram maps software-development procedures onto specialized agents. A human supplies a short requirement, such as a Flappy Bird game. The product manager turns it into a requirements document, the architect produces a system design, the project manager breaks the work into tasks, engineers write code, and QA checks the result. Each stage creates an artifact for the next role. The human returns for acceptance. The central idea is that explicit responsibilities and handoffs organize collaboration instead of leaving every agent to improvise the whole project.

Source: Sirui Hong et al., MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework, ICLR 2024. Figure 1.

Figure 2

Structured messages and executable feedback

On the left, agents publish structured messages into a shared pool and subscribe to information relevant to their roles. Messages identify their content, triggering action, sender, and recipient, helping the engineer find the design and task information needed for implementation. On the right, the engineer uses a role profile, memory, and previous artifacts to generate code. Execution then supplies feedback: errors trigger debugging and revision, while usable code is added to the project history. Requirements and design documents remain available during this loop so repairs can stay aligned with the original task.

Source: Sirui Hong et al., MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework, ICLR 2024. Figure 2.

Figure 3

Following the 2048 game through the agent team

This example follows a request for a 2048 game through concrete intermediate outputs. The product manager writes goals, user stories, and requirements; the architect specifies modules, interfaces, and a file list; the project manager creates implementation tasks; the engineer writes the files; and QA reviews the code. Clarification arrows connect roles when information is incomplete. The pictured code is an intermediate implementation artifact, not a complete standalone program. The final gameplay panel shows the intended handoff to a human user. The example makes the operating procedures visible as documents, diagrams, tasks, code, and review.

Source: Sirui Hong et al., MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework, ICLR 2024. Figure 3.

Figure 4

Reading the HumanEval and MBPP results

The blue bars show HumanEval and the purple bars show MBPP. Pass@1 measures the proportion of problems solved by one candidate solution under the reported evaluation. MetaGPT reaches 85.9% on HumanEval and 87.7% on MBPP. Without executable feedback, the corresponding values are 81.7% and 82.3%, so the full configuration improves them by 4.2 and 5.4 percentage points. This comparison supports the value of checking and revising generated programs in this experiment. A dash denotes an unreported result, not a zero score. These are the paper’s benchmark results, not current rankings or guarantees for arbitrary software projects.

Source: Sirui Hong et al., MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework, ICLR 2024. Figure 4.

Figure 5

Capabilities and the cost of adding specialized roles

The upper table is the paper’s feature comparison: check marks indicate that a framework provides a capability, not that it always performs it correctly. MetaGPT is marked as supporting requirements documents, technical designs, API interfaces, code generation, precompilation execution, role-based task management, and code review. The lower table removes or adds roles to examine their contribution. Moving from an engineer alone to all four listed roles raises reported expense from 0.915to0.915 to1.385, while human revision cost falls from 10 to 2.5 and the reported executability score rises from 1.0 to 4.0. The score is not a percentage. Adding roles changes both output and cost, and more generated lines do not necessarily mean better software. These comparisons describe the versions and experimental setting studied in the paper.

Source: Sirui Hong et al., MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework, ICLR 2024. Tables 2 and 3.

05

Architecture

  • The architecture is organized around role-playing agents connected by an SOP-driven workflow.
  • The product manager is responsible for requirement analysis.
  • The architect transforms requirements into a system design.
  • The project manager coordinates task decomposition and assignment.
  • Engineers generate implementation code.
  • The shared workspace contains intermediate artifacts that downstream roles consume.
  • The assembly line reduces direct all-to-all communication.
  • Structured communication is intended to make agent outputs more verifiable and less ambiguous.
  • The design can be extended by introducing additional specialized roles or SOP stages.

06

Dataset

  • MetaGPT evaluates generated software requirements and implementations rather than training on a conventional ML benchmark dataset.
  • The paper uses software-development task prompts as the primary input data.
  • HumanEval is used as a code-generation benchmark in the evaluation.
  • MBPP is also used to assess program synthesis capability.
  • The paper introduces SoftwareDev as a benchmark focused on complete software-generation tasks.
  • SoftwareDev contains 70 task descriptions covering different application requirements.
  • The benchmark evaluates whether an agent system can produce coherent multi-file software from requirements.
  • Evaluation includes both automated metrics and human judgments of generated software quality.

07

Training

  • MetaGPT is primarily an inference-time multi-agent framework and does not require training a new foundation model.
  • The main adaptation mechanism is prompt engineering around role-specific SOPs.
  • System prompts encode domain expertise and workflow constraints for each role.
  • Agents use pretrained LLMs to generate requirements, designs, task lists, and code.
  • The framework can therefore change collaboration behavior without changing model parameters.
  • Different underlying LLMs can be used as long as they can follow the required role and artifact instructions.
  • The paper's contribution is orchestration and workflow design rather than parameter-efficient fine-tuning.

08

Experiments

  • The experiments compare MetaGPT with prior multi-agent chat systems on software-development tasks.
  • The authors evaluate the ability to generate complete software from natural-language requirements.
  • SoftwareDev is used to test end-to-end software generation.
  • HumanEval and MBPP provide standardized code-generation checks.
  • The study examines both functional correctness and qualitative coherence.
  • Human evaluation considers whether the generated projects satisfy requirements and maintain consistency.
  • The experiments focus on whether SOP-driven role specialization reduces the weaknesses of unconstrained agent conversations.

09

Baselines

  • Baselines include ChatDev and other contemporary multi-agent LLM systems described in the paper.
  • Single-agent LLM generation provides a reference for the value of collaboration.
  • Chat-based multi-agent approaches are important baselines because they allow several agents to converse without MetaGPT's structured SOP pipeline.
  • Human-written software requirements provide the task input rather than synthetic benchmark-only prompts.
  • Standard code benchmarks such as HumanEval and MBPP provide complementary reference points.
  • The key comparison is therefore structured organization versus relatively free-form agent collaboration.

10

Results

  • MetaGPT generates more coherent software solutions than previous chat-based multi-agent systems in the reported SoftwareDev evaluation.
  • The framework achieves 85.9% Pass@1 on HumanEval.
  • It achieves 87.7% Pass@1 on MBPP in the reported evaluation.
  • On the SoftwareDev benchmark, the paper reports a 100% success rate under its project-level success criterion.
  • The system produces explicit intermediate artifacts such as PRDs and architecture designs.
  • Human evaluation favors MetaGPT for consistency between requirements, design, and implementation.
  • The results support the claim that structured SOPs reduce cascading errors compared with naive agent chaining.

11

Conclusion

  • MetaGPT demonstrates that organizational structure can be a first-class component of LLM-agent design.
  • SOPs turn human software-development practices into executable prompt workflows.
  • Role specialization gives agents explicit responsibilities and reduces unstructured conversation.
  • Intermediate artifacts create interfaces between stages of the workflow.
  • The assembly-line paradigm enables complex software generation through coordinated subtasks.
  • The reported results show improved coherence over prior chat-based multi-agent systems.
  • The broader lesson is that reliable agent systems need workflow constraints, verification, and domain-specific interfaces in addition to capable LLMs.

12

References

  • Primary paper: Sirui Hong et al., “MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework,” arXiv:2308.00352.
  • The paper was published at ICLR 2024.
  • Authors include Sirui Hong, Mingchen Zhuge, Jiaqi Chen, Xiawu Zheng, Yuheng Cheng, Ceyao Zhang, Jinlin Wang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, Chenyu Ran, Lingfeng Xiao, Chenglin Wu, and Jürgen Schmidhuber.
  • The project is available at https://github.com/geekan/MetaGPT.
  • Primary source: https://arxiv.org/abs/2308.00352.
  • The paper's central contribution is SOP-driven multi-agent collaboration for complex software-engineering tasks.

Continue reading

Related research

Browse all research