Agent-Editing World Model: Rethinking World Modeling for LLM Agents

Long-horizon agents are not only prediction systems; they are stateful decision processes. Every reasoning trace and tool action becomes part of the history used for later decisions. A wrong hypothesis can therefore become persistent state: an agent may treat an unverified lead as fact, keep an obsolete plan after contradictory evidence, or mistake partial progress for completion. Real observations do not automatically repair these mistakes because the agent still has to interpret them. The paper calls this failure mode task-state contamination. Existing language world models generally borrow an environment-centered objective: predict what a tool or environment will return after an action. AEWM asks a more agent-centered question: given what the agent currently knows and what it is about to do, will that decision help, explore productively, or reinforce a bad direction? This reframing makes the world model an intervention mechanism rather than a simulator.

The core problem is persistent decision error under partial observability. Let the visible interaction history be h_t and the agent propose reasoning r-hat_t and action a-hat_t. The proposal is not merely an ephemeral thought: once executed and observed, it becomes part of h_{t+1}. If the proposal embeds an unsupported assumption, subsequent steps inherit it. The system therefore needs a way to judge a proposal before execution and, when necessary, replace both the reasoning and action rather than merely attaching a critique. The paper also argues that observation prediction is a poor fit for many tool environments because search rankings, web content, filesystems, runtime state, and test outcomes are difficult to predict exactly and are already available from real execution.

Paper-derived background: conventional world models learn action-conditioned environment dynamics; language world models extend this idea to tool or web observations. AEWM instead connects to process supervision, revision learning, ReAct-style tool use, and world models for web/code agents. The paper distinguishes three decision roles. CRITICAL decisions close an important gap or perform a necessary state change. EXPLORATORY decisions reduce uncertainty or test a plausible branch and therefore should not be suppressed merely because they are not on the shortest path. NOISY decisions have little expected progress, repeat unproductive behavior, violate constraints, or reinforce an incorrect direction. External research: the surrounding literature includes observation-predictive language world models, web agents with world models, Reflexion-style verbal feedback, and process verifiers. AEWM differs by making downstream decision effect and state editing the central modeling target.

The method has three linked stages. First, Action Judge data are synthesized from successful verified trajectories. An annotation agent labels individual turns using the corresponding observation and later task outcome, while filtering checks label consistency, action-observation alignment, action validity, and grounding in information visible before execution. Second, State Revision data are generated by proposing a task step, using an Action Judge checkpoint to find NOISY proposals, generating a replacement from the same history, executing it in the real environment, and retaining it only when the new step produces substantive progress. Third, EditAct runs online: the base agent proposes a reasoning-action pair; AEWM judges it; CRITICAL and EXPLORATORY proposals are retained, while NOISY proposals are replaced by State Revision; the selected action is then executed in the real environment.

The AEWM architecture is an intervention loop around an ordinary agent. The base agent proposes (r-hat_t, a-hat_t) from the current history h_t. AEWM Action Judge reads the pre-execution state and emits one of three decision-effect labels. If the label is CRITICAL or EXPLORATORY, the proposal proceeds unchanged. If it is NOISY, State Revision generates (r-tilde_t, a-tilde_t) from the same visible history. Only the selected action is executed. The real environment returns observation o_t, which is appended to the history before the next agent turn. This matters because the model edits the state that future reasoning will consume rather than merely producing a side-channel score. Figure 2 in the paper shows the training-data synthesis, AEWM training, EditAct inference, and AEWM-RFT loop.

The principal downstream baseline is ReAct, which reasons and acts against real tools but does not selectively edit its current reasoning-action continuation. Step-level Best@3 samples three candidates at each decision and uses a verifier to select one; trajectory-level Best@3 generates up to three complete trajectories and submits the verifier-selected result. These baselines test whether AEWM is merely benefiting from extra candidate generation. The transfer baseline Self-RFT trains Qwen3.5-35B-A3B on its own verified trajectories, whereas AEWM-RFT learns from trajectories improved by EditAct. Action Judge is separately compared with strong frontier models.

Table 3 isolates the source of the gains. On BrowseComp, Terminal-Bench 2.0, and Doc2Repo, full AEWM scores 48.1, 48.3, and 48.9. Random gating is lower at 44.1, 40.8, and 43.9, supporting learned intervention selection. Agent resampling and AEWM Hint are also below the full system, supporting direct state editing over generic regeneration or guidance. Action-only and reasoning-only revision do not match joint revision. Self-WM and DeepSeek-V4-Pro WM are below AEWM on the three reported tasks. Training ablations show that full mid-training plus SFT beats SFT-only by 0.8, 6.4, and 4.3 points, and beats mid-training-only by 0.3, 4.1, and 1.9 points on the same benchmarks. Figure 5 adds a behavioral signal: the longest search-only sequence falls from 16.9 to 13.2 while distinct requested pages rise from 7.7 to 10.1 with AEWM.

The paper does not provide a standalone limitations section, so limitations are separated into paper evidence and engineering interpretation. Paper evidence: Qwen3.5-Plus gains are strong on BrowseComp but limited on Terminal and SWE, which the authors attribute to a capacity gap between the stronger agent and AEWM. A learned gate can also make false interventions: suppressing a useful action is costly, while missing a noisy action leaves contamination unresolved. Evaluation covers Search, Terminal, and SWE rather than GUI, robotics, multimodal, or multi-agent settings. Engineering interpretation: EditAct adds online model calls, so latency, memory, token cost, and recovery overhead should be measured before production deployment; the paper does not provide a complete wall-clock or cost analysis for this extra loop. The 3,000-decision judgment benchmark is carefully constructed but depends on trajectory-derived labels and annotation/filtering quality.

AEWM reframes world modeling for LLM agents around the state that controls future decisions. Its strongest idea is not simply adding a critic; it is to distinguish useful exploration from harmful continuation, revise reasoning and action together when necessary, and then obtain the next observation from the real environment. Results support this design across Search, Terminal, and SWE, with consistent gains across three agent scales and transfer through AEWM-RFT. The production lesson is to treat task state as an explicit reliability surface. If stale assumptions, failed plans, or partial completion can persist in history, an intervention layer should be evaluated on whether it improves the committed trajectory, not only whether it produces a plausible critique.

Primary: Shuang Sun et al., Agent-Editing World Model: Rethinking World Modeling for LLM Agents, arXiv:2609.28416, 2026. Related references used by the paper include ReAct, Reflexion, General Agents Need World Models, Web Agents with World Models, ECHO: Terminal Agents Learn World Models for Free, SWE-Master, CalibForge, and the cited Search/Terminal/SWE benchmarks. External research checked during this run includes the official arXiv paper record and the public RUC AI Box repository listing for the implementation.