Recursive Self-Improvement
An AI improving its own capabilities iteratively, potentially leading to a rapid intelligence explosion.
As defined in Ilya Sutskever – We're moving from the age of scaling to the age of research
Other definitions
An AI system's ability to independently modify and improve its own code and capabilities without external oversight or human control, potentially leading to rapid capability gains.
What Is an AI Anyway? | Mustafa Suleyman | TEDA process in which AI systems improve the AI research or models that create them, potentially accelerating capability growth.
Sarah Guo - Funding the Frontier - [Invest Like the Best, EP.489]
Where it comes up · 25
- Dwarkesh Patel Podcast - Paul Christiano – We could be 12 months away from AIs that can overthrow the government4 claims
- Dwarkesh Patel Podcast - The avalanche that hit math is coming for AI development4 claims
- Ilya Sutskever – We're moving from the age of scaling to the age of research3 claims
- 116 - Ai Racing Toward The Brink3 claims
- Ask Me Anything 13 claims
- Ex-Google CEO: What Artificial Superintelligence Will Actually Look Like w/ Eric Schmidt & Dave B3 claims
- Capital, Compute & AI Scaling2 claims
- Dwarkesh Patel Podcast - Alex Imas and Phil Trammell – What remains scarce after AGI?2 claims
and 17 more
Most important claims · 10 of 50Search all claims for “Recursive Self-Improvement” →
- Algorithms like AlphaGo's move 37 and Noam Brown's poker bot's overbetting show that AI can generate moves outside human knowledge, suggesting that LLMs bounded by training data may need a way to escape their upfront data limits.Capital, Compute & AI Scaling
- Just as evolution programmed humans only to reproduce yet cannot see or anticipate anything we actually do (conversations, building machines, choosing monasticism or surfing over reproduction), a sufficiently intelligent AI that rewrites its own code to become more intelligent will find cognitive horizons its builders cannot foresee; we are therefore guaranteed to lose sight of what it can understand and care about.324 - Debating The Future Of Ai
- The danger follows tautologically from the words 'superhuman' and 'general' intelligence: building something more intelligent than us that revises its own code is like building something stronger than you and then wrestling with it; once it exists you have already lost the game (as in chess against a superior engine), so there is no reason to believe it will be necessarily benign simply because we built it.324 - Debating The Future Of Ai
- Once you accept that intelligence is not substrate-dependent and we will build genuine intelligence in machines, you should expect deception, manipulation, and the formation of unforeseeable instrumental goals—especially when building something smarter than us, or even something only as smart as us that runs a million times faster, since speed alone is enough to completely outclass humans.469 - Escaping An Anti-human Future
- There are two parallel fast exponentials—one is the capability of the model, the other is the diffusion of the model into the economy—and diffusion is much faster than any previous technology but not instant; it is bounded by the need to close the loop (change management, security permissions, rewriting old software, telling the model what to do), so the future is a 'middle world' of extremely fast but not instantaneous progress.Dwarkesh Patel Podcast - Dario Amodei — "We are near the end of the exponential"
- The single most important guardrail is banning closed-loop recursive self-improvement—where an AI runs its own experiments and rewrites itself a million times—because that is an event horizon with no way to know what comes out the other side; this should be illegal with jail time, since there is no reason to believe anyone could do it safely.469 - Escaping An Anti-human Future
- Certain AI capabilities should be explicitly prohibited: (1) use for electioneering and counterfeit digital people, (2) autonomous weapon systems, and (3) recursive self-improvement where AI updates its own code to pursue its own objectives, because these capabilities are dangerous and we have not yet developed sufficient guardrails.Mustafa Suleyman & Yuval Noah Harari -FULL DEBATE- What does the AI revolution mean for our future?
- The critical inflection point is when AI can fully substitute for a machine-learning researcher, because at that moment the world effectively gains hundreds of millions of AI researchers working on the next generation plus millions of AI scientists on other problems, producing a discontinuous leap in capability—an intelligence explosion—plausibly arriving in the early 2030s with large error bars.467 - Ea Ai And The End Of Work
- As AI development itself becomes increasingly automated, automation increases the quality and quantity of labor applied to AI R&D, so capability changes that would normally take four to six years could be compressed into a very short period.Dwarkesh Patel Podcast - Paul Christiano – We could be 12 months away from AIs that can overthrow the government
- There are specific red-line criteria where AI systems should be automatically shut down: recursive self-improvement (the system improving itself without human knowledge), direct access to weapons, and exfiltration/self-replication without permission.The AI Revolution Is Underhyped | Eric Schmidt | TED