SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering
John Yang12 min readResearch
FOUNDATIONALAI AgentsAdvanced
Key takeaway
Foundational work on coding agents and agent-computer interfaces.
Updated on Oct 3, 2026
01
Abstract
- SWE-agent is an autonomous software-engineering agent designed to resolve real GitHub issues.
- Its central contribution is the Agent-Computer Interface (ACI), an interface tailored to how language models search, edit, and test repositories.
- The agent uses a language model to interact with a repository through a controlled set of shell and file-editing actions.
- The system is evaluated primarily on SWE-bench, a benchmark of real GitHub issues.
- SWE-agent is designed around iterative inspection, modification, testing, and repair.
- The paper argues that interface design can be as important as model capability for software agents.
- The reported system substantially improves automated issue-resolution performance compared with prior approaches.
02
Introduction
- Software engineering is a long-horizon task requiring navigation of unfamiliar codebases.
- A model must locate relevant files, understand surrounding code, make targeted edits, run tests, and interpret failures.
- Generic computer interfaces expose too many low-level operations or make important repository information difficult to access.
- The authors argue that LLMs need an interface optimized for software engineering rather than a generic terminal alone.
- SWE-bench provides realistic issue descriptions paired with real repository states and developer patches.
- Success requires producing a repository change that passes the benchmark's tests, not merely generating plausible code.
- SWE-agent therefore co-designs the agent policy and the computer interface.
03
Problem
- The central problem is how to enable an LLM to reliably solve real software-engineering issues in existing repositories.
- Repository-scale tasks require search, navigation, editing, execution, and debugging.
- Long context cannot simply contain an entire repository, so the agent must retrieve relevant code incrementally.
- The agent needs feedback from tests and commands to validate hypotheses.
- Poor interface choices can cause the model to waste context or take ineffective actions.
- The paper tests whether an ACI specifically designed for coding improves issue resolution.
- The benchmark measures end-to-end task success on real-world GitHub issues.
04
Methodology
- SWE-agent follows an iterative observation → reasoning → action → feedback loop.
- The model receives the issue description and repository context.
- It can search for symbols or text to locate relevant code.
- It can inspect selected portions of files.
- It can apply edits through the ACI rather than generating an entire repository patch in one response.
- It can run tests or other shell commands after editing.
- Test failures become evidence for subsequent debugging steps.
- The loop continues until the issue is resolved or the interaction budget is exhausted.
- The agent's context therefore contains only the repository information needed for the current reasoning step rather than the entire codebase.
Figure 1
05
Architecture
- The ACI includes specialized commands for repository navigation and file manipulation.
- Search operations help locate relevant code without loading entire repositories.
- File-viewing operations expose bounded regions of source files.
- Editing commands allow localized changes while preserving surrounding repository state.
- Shell execution allows tests and diagnostic commands to be run.
- The interface includes safeguards intended to make actions easier for the LLM to use correctly.
- The paper emphasizes that command design, feedback formatting, and context management jointly determine agent performance.
- The environment is executed in a containerized setting so repository state and tool behavior are controlled.
- This architecture turns software development into an interactive tool-use problem rather than one-shot code generation.
06
Dataset
- The primary benchmark is SWE-bench, which contains real GitHub issues from open-source repositories.
- SWE-bench instances pair an issue description with a repository snapshot and an associated developer patch.
- The benchmark spans multiple Python repositories and real historical issue reports.
- SWE-agent evaluates issue resolution rather than only unit-test generation.
- SWE-bench's test suite is used to determine whether a generated change resolves the issue.
- The evaluation uses the SWE-bench test-based success criterion.
- The paper also reports analysis on subsets of SWE-bench to understand issue difficulty and failure modes.
- The benchmark is particularly valuable because tasks originate from real developer workflows.
07
Training
- SWE-agent uses an LLM as the reasoning controller and the ACI as its action interface.
- The paper evaluates GPT-4-class models and other language-model configurations.
- The agent is not trained from scratch for each repository.
- Repository-specific adaptation occurs through in-context reasoning, search, editing, and test feedback.
- The ACI itself encodes software-engineering knowledge through its available operations and feedback format.
- This means some of the system's gains come from interface engineering rather than parameter updates.
- The agent can use test execution as an external feedback signal during inference.
08
Experiments
- The main evaluation measures the percentage of SWE-bench issues resolved successfully.
- The paper compares SWE-agent with prior autonomous software-engineering systems.
- The study evaluates the effect of different ACI designs and interaction strategies.
- Error analysis examines failures in repository navigation, patch generation, test execution, and reasoning.
- The benchmark includes issues that require changes across multiple files and complex dependency reasoning.
- The evaluation is performed in reproducible repository environments.
- The authors analyze trajectories to understand why agents fail even when the underlying LLM is strong.
09
Baselines
- SWE-agent is compared with earlier autonomous coding agents and SWE-bench systems.
- A key conceptual baseline is a generic shell or computer interface without the SWE-agent-specific ACI.
- The comparison tests whether specialized interface actions improve repository-level problem solving.
- GPT-4-based configurations provide a strong model baseline for measuring interface effects.
- SWE-bench's developer patch is used as the reference solution for benchmark instances, while tests determine functional success.
- Human-written repository fixes represent the practical target rather than a generated-code similarity score.
10
Results
- SWE-agent achieves 12.47% resolved issues on the full SWE-bench test set in the paper's main reported result.
- This result was state of the art at the time of the paper's submission.
- The paper reports 18.0% performance on a filtered subset referred to as SWE-bench Lite in later evaluations of the system.
- The main result demonstrates that a relatively simple interaction loop can solve a non-trivial fraction of real GitHub issues.
- The ACI contributes substantially to performance by helping the model locate and manipulate relevant code.
- Test feedback allows the agent to repair patches instead of stopping after its first edit.
- The results show that repository interaction quality matters alongside raw language-model capability.
11
Conclusion
- SWE-agent demonstrates that the computer interface itself can be a major source of agent capability.
- The ACI provides focused search, navigation, editing, and execution primitives.
- Iterative test-driven interaction enables the agent to refine changes using real repository feedback.
- SWE-bench provides a realistic evaluation target because issues come from real open-source development.
- The 12.47% main reported resolution rate established a strong result for autonomous issue resolution at the time.
- The work motivates co-designing language models, tools, and interfaces for software agents.
- The broader lesson is that effective coding agents need repository-aware interaction loops, not only larger language models.
12
References
- Primary paper: John Yang et al., “SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering,” arXiv:2405.15793.
- The paper was published at NeurIPS 2024.
- The project is available at https://github.com/SWE-agent/SWE-agent.
- Primary source: https://arxiv.org/abs/2405.15793.
- The evaluation centers on SWE-bench and real GitHub issue resolution.
- The paper's key contribution is the Agent-Computer Interface designed specifically for software-engineering agents.
Source trail
Primary papers, implementation pages, and external source records used to ground this explainer.
Continue reading
Related research
AI Agents
GitSwarm
GitSwarm turns inference-time work into a persistent, branchable body of reusable contributions that later agents can build on.
AI AgentsLongHarness
LongHarness helps agents reason over 100M-token document collections by keeping evidence and conclusions as explicit, revisable states that guide one another.
AI AgentsGenerative Agents
An agent architecture combining memory, retrieval, reflection, and planning to simulate believable long-horizon behavior.