Research

SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering

John Yang12 min readResearch
FOUNDATIONALAI AgentsAdvanced

Key takeaway

Foundational work on coding agents and agent-computer interfaces.

Updated on Oct 3, 2026

01

Abstract

  • SWE-agent is an autonomous software-engineering agent designed to resolve real GitHub issues.
  • Its central contribution is the Agent-Computer Interface (ACI), an interface tailored to how language models search, edit, and test repositories.
  • The agent uses a language model to interact with a repository through a controlled set of shell and file-editing actions.
  • The system is evaluated primarily on SWE-bench, a benchmark of real GitHub issues.
  • SWE-agent is designed around iterative inspection, modification, testing, and repair.
  • The paper argues that interface design can be as important as model capability for software agents.
  • The reported system substantially improves automated issue-resolution performance compared with prior approaches.

02

Introduction

  • Software engineering is a long-horizon task requiring navigation of unfamiliar codebases.
  • A model must locate relevant files, understand surrounding code, make targeted edits, run tests, and interpret failures.
  • Generic computer interfaces expose too many low-level operations or make important repository information difficult to access.
  • The authors argue that LLMs need an interface optimized for software engineering rather than a generic terminal alone.
  • SWE-bench provides realistic issue descriptions paired with real repository states and developer patches.
  • Success requires producing a repository change that passes the benchmark's tests, not merely generating plausible code.
  • SWE-agent therefore co-designs the agent policy and the computer interface.

03

Problem

  • The central problem is how to enable an LLM to reliably solve real software-engineering issues in existing repositories.
  • Repository-scale tasks require search, navigation, editing, execution, and debugging.
  • Long context cannot simply contain an entire repository, so the agent must retrieve relevant code incrementally.
  • The agent needs feedback from tests and commands to validate hypotheses.
  • Poor interface choices can cause the model to waste context or take ineffective actions.
  • The paper tests whether an ACI specifically designed for coding improves issue resolution.
  • The benchmark measures end-to-end task success on real-world GitHub issues.

04

Methodology

  • SWE-agent follows an iterative observation → reasoning → action → feedback loop.
  • The model receives the issue description and repository context.
  • It can search for symbols or text to locate relevant code.
  • It can inspect selected portions of files.
  • It can apply edits through the ACI rather than generating an entire repository patch in one response.
  • It can run tests or other shell commands after editing.
  • Test failures become evidence for subsequent debugging steps.
  • The loop continues until the issue is resolved or the interaction budget is exhausted.
  • The agent's context therefore contains only the repository information needed for the current reasoning step rather than the entire codebase.
Figure 1

05

Architecture

  • The ACI includes specialized commands for repository navigation and file manipulation.
  • Search operations help locate relevant code without loading entire repositories.
  • File-viewing operations expose bounded regions of source files.
  • Editing commands allow localized changes while preserving surrounding repository state.
  • Shell execution allows tests and diagnostic commands to be run.
  • The interface includes safeguards intended to make actions easier for the LLM to use correctly.
  • The paper emphasizes that command design, feedback formatting, and context management jointly determine agent performance.
  • The environment is executed in a containerized setting so repository state and tool behavior are controlled.
  • This architecture turns software development into an interactive tool-use problem rather than one-shot code generation.

06

Dataset

  • The primary benchmark is SWE-bench, which contains real GitHub issues from open-source repositories.
  • SWE-bench instances pair an issue description with a repository snapshot and an associated developer patch.
  • The benchmark spans multiple Python repositories and real historical issue reports.
  • SWE-agent evaluates issue resolution rather than only unit-test generation.
  • SWE-bench's test suite is used to determine whether a generated change resolves the issue.
  • The evaluation uses the SWE-bench test-based success criterion.
  • The paper also reports analysis on subsets of SWE-bench to understand issue difficulty and failure modes.
  • The benchmark is particularly valuable because tasks originate from real developer workflows.

07

Training

  • SWE-agent uses an LLM as the reasoning controller and the ACI as its action interface.
  • The paper evaluates GPT-4-class models and other language-model configurations.
  • The agent is not trained from scratch for each repository.
  • Repository-specific adaptation occurs through in-context reasoning, search, editing, and test feedback.
  • The ACI itself encodes software-engineering knowledge through its available operations and feedback format.
  • This means some of the system's gains come from interface engineering rather than parameter updates.
  • The agent can use test execution as an external feedback signal during inference.

08

Experiments

  • The main evaluation measures the percentage of SWE-bench issues resolved successfully.
  • The paper compares SWE-agent with prior autonomous software-engineering systems.
  • The study evaluates the effect of different ACI designs and interaction strategies.
  • Error analysis examines failures in repository navigation, patch generation, test execution, and reasoning.
  • The benchmark includes issues that require changes across multiple files and complex dependency reasoning.
  • The evaluation is performed in reproducible repository environments.
  • The authors analyze trajectories to understand why agents fail even when the underlying LLM is strong.

09

Baselines

  • SWE-agent is compared with earlier autonomous coding agents and SWE-bench systems.
  • A key conceptual baseline is a generic shell or computer interface without the SWE-agent-specific ACI.
  • The comparison tests whether specialized interface actions improve repository-level problem solving.
  • GPT-4-based configurations provide a strong model baseline for measuring interface effects.
  • SWE-bench's developer patch is used as the reference solution for benchmark instances, while tests determine functional success.
  • Human-written repository fixes represent the practical target rather than a generated-code similarity score.

10

Results

  • SWE-agent achieves 12.47% resolved issues on the full SWE-bench test set in the paper's main reported result.
  • This result was state of the art at the time of the paper's submission.
  • The paper reports 18.0% performance on a filtered subset referred to as SWE-bench Lite in later evaluations of the system.
  • The main result demonstrates that a relatively simple interaction loop can solve a non-trivial fraction of real GitHub issues.
  • The ACI contributes substantially to performance by helping the model locate and manipulate relevant code.
  • Test feedback allows the agent to repair patches instead of stopping after its first edit.
  • The results show that repository interaction quality matters alongside raw language-model capability.

11

Conclusion

  • SWE-agent demonstrates that the computer interface itself can be a major source of agent capability.
  • The ACI provides focused search, navigation, editing, and execution primitives.
  • Iterative test-driven interaction enables the agent to refine changes using real repository feedback.
  • SWE-bench provides a realistic evaluation target because issues come from real open-source development.
  • The 12.47% main reported resolution rate established a strong result for autonomous issue resolution at the time.
  • The work motivates co-designing language models, tools, and interfaces for software agents.
  • The broader lesson is that effective coding agents need repository-aware interaction loops, not only larger language models.

12

References

  • Primary paper: John Yang et al., “SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering,” arXiv:2405.15793.
  • The paper was published at NeurIPS 2024.
  • The project is available at https://github.com/SWE-agent/SWE-agent.
  • Primary source: https://arxiv.org/abs/2405.15793.
  • The evaluation centers on SWE-bench and real GitHub issue resolution.
  • The paper's key contribution is the Agent-Computer Interface designed specifically for software-engineering agents.

Continue reading

Related research

Browse all research