self-play
A training method where a system plays against itself many times to generate effectively infinite free training data, viable only in closed-world games with fixed rules.
As defined in 184 | Gary Marcus on Artificial Intelligence and Common Sense
Where it comes up · 17
- Ilya Sutskever: Deep Learning | Lex Fridman Podcast #943 claims
- Ilya Sutskever – We're moving from the age of scaling to the age of research3 claims
- #416: The Relentless Missionary Creating AGI: Demis Hassabis3 claims
- Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history3 claims
- Capital, Compute & AI Scaling2 claims
- Dwarkesh Patel Podcast - Noam Brown – Agent swarms, alignment, & recursive self-improvement2 claims
- AI 2027: month-by-month model of intelligence explosion — Scott Alexander & Daniel Kokotajlo1 claim
- Andrej Karpathy — AGI is still a decade away1 claim
and 9 more
Most important claims · 10 of 27Search all claims for “self-play” →
- Hassabis's team believed that pattern-matching human play would not reach the level of the top human player, so the system needed to discover novel, non-human moves; the early version learned human strategies, then discarded some that could be counteracted, and became alien in style as it strengthened.#416: The Relentless Missionary Creating AGI: Demis Hassabis
- AlphaGo Zero was trained exclusively through self-play using random moves and a reward signal rather than expert human games, and it outclassed its predecessor by a wide margin, discovering strategies unknown to human players.#416: The Relentless Missionary Creating AGI: Demis Hassabis
- Algorithms like AlphaGo's move 37 and Noam Brown's poker bot's overbetting show that AI can generate moves outside human knowledge, suggesting that LLMs bounded by training data may need a way to escape their upfront data limits.Capital, Compute & AI Scaling
- AlphaZero's ability to learn entirely through self-play—generating instances of itself to play billions of simulated games—proved more valuable than bootstrapping from centuries of accumulated human game knowledge, because human expert games represent only a minor subset of the vast possible state space.332 - Can We Contain Artificial Intelligence
- LLMs lack two powerful multi-agent ideas: culture (a growing repertoire of knowledge they write down and share, like one LLM writing a book for others) and self-play (LLMs generating progressively harder problems for each other to solve, as AlphaGo did against itself); both remain unclaimed research realms, which is one reason the field is still early.Andrej Karpathy — AGI is still a decade away
- The universe gradually became more complex because in a complex environment, the smarter an organism is, the more successful it is, so it has more offspring and itself becomes the environment for others, who in turn gain an incentive to get smarter—creating an ever more complex environment; this is why fish in simple environments are less smart, and it parallels DeepMind's self-play where a program playing against copies of itself keeps getting stronger.94 - Frontiers Of Intelligence
- RL reasoning training only works in verifiable domains (math, code) where answers can be checked; the next 'move 37' breakthrough will likely come from computer use or robotics rather than science, because those provide infinitely verifiable sandbox playgrounds where models can self-play, fail, and learn at scale like a baby learning its body.Deepseek Dylan Patel Nathan Lambert
- Because RL training of LLMs gives a model a problem and asks it to solve it, unlike self-play with an infinite curriculum in AlphaZero, once models become smart enough that problems are too easy, they stop learning, making it a plausible scenario that LLM progress becomes much harder.Dwarkesh Patel Podcast - Noam Brown – Agent swarms, alignment, & recursive self-improvement
- Deep learning systems excel at closed worlds where rules are fixed (ideally unchanged for 2000 years), you can self-play to generate infinite free data, and nothing unforeseen happens; they fail to generalize to the real world where rules vary, data is costly, and unknown situations like an unexpected snowstorm arise.184 | Gary Marcus on Artificial Intelligence and Common Sense
- When human-generated data runs out, models that generate their own training data through self-play and self-critique, as AlphaGo did, will keep scaling, and this technique is already showing promise in language.Gavin Uberti - Real-Time AI & The Future of AI Hardware - [Invest Like the Best, EP.356]