Module-by-module ablation of NLAH found that self-evolution was the only consistently helpful module (+4.8 on SWE-Bench, +2.7 on OS World) via an acceptance-gated attempt loop that stays narrow until failure signals justify broadening, while verifiers hurt performance (-0.8 and -8.4) and multi-candidate search was actively harmful (-2.4 and -5.6).

factualpending

Speaker

Unidentified Speaker — Rethinking AI Agents: The Rise of Harness Engineering [Xxuxg8PcBvc]

Evidence Quote

“Then the module-by-module ablation found something stranger. Self-evolution was the only consistently helpful module, plus 4.8 on SWE-Bench, plus 2.7 on OS World. Via an acceptance-gated attempt loop that stays narrow until failure signals justify broadening. Verifiers actively hurt, minus 0.8 and minus 8.4. Multi-candidate search, minus 2.4 and minus 5.6.”

Source

Rethinking AI Agents: The Rise of Harness Engineering— PY
Created: 8/12/2026, 2:27:33 AM

Your notes

Loading notes...