MachinoAI Research
RRSI: Regularized Recursive Self-Improvement of Agent Harnesses
Regularize the search trajectory used to evolve an agent harness.
TL;DR
RRSI adds proposal-side sparsity, evidence-aware exploration, leakage screening, noise-aware acceptance, cost-aware acceptance and pruning; it improves held-out performance and reduces policy-token use.
Why It Matters
Agent capability increasingly depends on the system around the model, so harness optimization is a first-class production problem.
Research Brief
The shortest useful explanation.
RRSI adds proposal-side sparsity, evidence-aware exploration, leakage screening, noise-aware acceptance, cost-aware acceptance and pruning; it improves held-out performance and reduces policy-token use.
Core Explanation
RRSI regularizes recursive improvement of LLM agent harnesses to reduce benchmark overfitting and improve transfer while lowering policy-token cost.
Why It Matters
Agent capability increasingly depends on the system around the model, so harness optimization is a first-class production problem.
RRSI overview
Original paper figure.
Section 01
The Problem
01 THE PROBLEM
Agent behavior depends on prompts, control flow, tools, memory and context management. Manual tuning is slow; automated harness evolution can overfit the evolve set, chase noise and grow cost.
Section 02
The Big Idea
02 THE BIG IDEA
RRSI regularizes the evolution process instead of restricting the reachable harness space: proposal-side controls constrain edits and selection-side controls constrain what persists.
Section 03
How It Works
03 HOW IT WORKS
Evaluate current harness, summarize feedback, propose candidates, screen for leakage, evaluate candidates, apply noise/cost gates and prune unproductive components; then test the final harness on held-out benchmarks.
Section 04
The Math
04 THE MATH
The paper formalizes expected score and policy-token cost, an annealed edit budget, a noise-adjusted floor and a cost-aware acceptance inequality.
Section 05
Architecture
05 ARCHITECTURE
Frozen backbone plus editable harness, proposer/analyst, leakage critic, selection gates and held-out evaluation. The original Figure 2 shows the proposal-side and selection-side controls.
Section 06
Experiments
06 EXPERIMENTS
Eight benchmarks: Terminal-Bench 2.1, SWE-bench Verified, Harvey LAB, JobBench, GDPval, APEX-Agents, EngDesign and Frontier-Eng. Reported gains include +6.0 Terminal-Bench, +4.9 EngDesign, +1.1 Harvey evolve, +1.8 SWE-bench Verified, +2.3 Harvey held-out, +3.5–4.7 JobBench/GDPval/APEX and +4.3 Medal points (24.3% relative) Frontier-Eng. Exact Table 1: H0 [89.4,86.9,36.0,48.8,34.2]; RRSI [90.5,89.2,40.7,52.3,37.9] for Harvey Evolve, Harvey ID Held-out, JobBench, GDPval, APEX.
Section 07
What We Learned
07 WHAT WE LEARNED
RRSI improves held-out splits, with up to 4.7 points OOD and up to 22.9% over average prior baselines. The abstract reports 30% fewer policy tokens; the project page reports -36% tokens per trial. Naive evolution fails through benchmark fitting, noise chasing and complexity accumulation.
Section 08
What Came Next
08 WHAT CAME NEXT
The paper compares Meta-Harness, AHE, TTHE and HarnessX and cites prior harness-evolution and recursive-self-improvement work.
Section 09
Engineering Takeaways
ENGINEERING TAKEAWAYS
Keep evolution and evaluation separate; log harness diffs with score/cost deltas; add leakage checks; use conservative acceptance under stochastic evaluation; prune components whose contribution disappears. Watch memorization, noisy winners, tool bloat and rising token cost.
Formulas
Score and cost
Expected task score and policy-token cost.
Annealed edit budget
Attributable edit budget decreases over rounds.
Cost-aware acceptance
Additional cost must be justified by score gain.
Noise-adjusted floor
Candidate must clear the empirical noise floor.
Related Research
Generative Agents
An agent architecture combining memory, retrieval, reflection, and planning to simulate believable long-horizon behavior.
AI AgentsMIRA
MIRA is an autonomous medical agent evaluated in a sandboxed EHR workflow with tools for diagnosis, testing, treatment, and admission decisions.
AI AgentsAgentic Reasoning
A tool-using agent framework that extends LLM reasoning with web search, coding, and structured reasoning memory for deep research tasks.