Self-improving AI agents will need to fix their own code. And this paper from top US+China labs, shows coding agents miss most such bugs but improve with lessons from past fixes. that real bugs in agent harnesses, can be automatically turned into a growing set of runnable tests. An agent's own code is everything around the model: tool calls, memory, and prompts. Its bugs depend on live model calls, which makes them hard to recreate and test. So the researchers built AgentBug-Smith, which turns real GitHub bug reports into runnable tests. The result is a 200-bug benchmark that keeps growing. The best of 3 coding agents fixed just 9% of those bugs, versus about 40% reported on regular software bugs. A short guide of lessons from past fixes lifted an agent from 1 to 6 correct fixes on 79 unseen bugs. Before trusting a coding agent with your agent's code, try it on bugs you've already fixed.

Oct 3, 2026 · 4:31 AM UTC

7
15
42
3,505
– arxiv.org/abs/2609.37864 Title: "AgentBug-Smith: Automatically Reproducing Real-World Harness Bugs in Agentic Systems"

AgentBug-Smith: Automatically Reproducing Real-World Harness Bugs...

Agent harness bugs exhibit unique characteristics and remain challenging for state-of-the-art software agents to repair. Progress in this area is further hindered by existing benchmarks, which...

arxiv.org
2
4
870
Sort replies: Relevant Recent Liked
Replying to @rohanpaul_ai
Lessons from past fixes matches what I see daily. The agent that builds my product kept repeating the same mistakes until every fix got a dated note in a file it reads first. Half my repo is now diary entries for a robot.
12