emergent capabilities
Abilities a model exhibits that were not explicitly engineered or directly trained, arising from scale and compositional generalization.
As defined in Fully autonomous robots are much closer than you think – Sergey Levine
Other definitions
Abilities that suddenly appear in large AI models at certain scales, not explicitly programmed but arising from the model's training and complexity
AI Snake Oil: What Artificial Intelligence Can Do, What It Can’t, and How to Tell the Difference
Where it comes up · 9
- AI Snake Oil: What Artificial Intelligence Can Do, What It Can’t, and How to Tell the Difference2 claims
- This is not the AI we were promised | The Royal Society2 claims
- Fully autonomous robots are much closer than you think – Sergey Levine2 claims
- Generative AI in a Nutshell - how to survive and thrive in the age of AI1 claim
- Capital, Compute & AI Scaling1 claim
- Howie Liu - Building Airtable - [Invest Like the Best, EP.375]1 claim
- Roman Yampolskiy1 claim
- AI Timelines and Human Psychology (with Sarah Hastings-Woodhouse)1 claim
and 1 more
Most important claims · 10 of 12Search all claims for “emergent capabilities” →
- Neural nets and LLMs represent the ultimate meta-approach to software development, because instead of hand-coding every if-this-then-that rule, developers train the system to perform the behavior, and LLMs show emergent capabilities even model researchers did not anticipate.Howie Liu - Building Airtable - [Invest Like the Best, EP.375]
- Large language models initially functioned as limited statistical machines (word predictors) with restricted practical utility, but as models became larger and trained on more data, they developed emergent capabilities—unexpected abilities that surprised even developers—including creative writing, poetry, high-quality code generation, strategic discussion, legal and medical advice, coaching, and teaching.Generative AI in a Nutshell - how to survive and thrive in the age of AI
- Scaling up foundation models on larger chunks of the internet and expecting emergent capabilities to emerge without domain-specific engineering has run out as a productive research direction because the new capabilities to learn are tacit knowledge, which requires active deployment and feedback loops, not passive training on internet text.AI Snake Oil: What Artificial Intelligence Can Do, What It Can’t, and How to Tell the Difference
- It is hard to know ex ante what a scaled model like GPT-5 would deliver, because scaling laws only promise lower training loss and next-token accuracy, not a clearly articulated emergent capability, so the signal to watch is a lab declaring the model's accuracy warrants continued log-linear scaling.Capital, Compute & AI Scaling
- Emergent capabilities in LLMs arise not merely from internet data abundance but from compositional generalization: once generalization reaches a level, the model can combine known elements in novel ways (e.g. writing a recipe in the International Phonetic Alphabet despite never seeing that combination), and the same compositionality emerges in robots at scale.Fully autonomous robots are much closer than you think – Sergey Levine
- Emergent capabilities in AI refer to capabilities the model demonstrates that were not explicitly trained into it or designed by engineers—the model acquired the capability without deliberate instruction.This is not the AI we were promised | The Royal Society
- GPT-3, released by OpenAI in June 2020, represented a breakthrough in large language models, demonstrating emergent capabilities such as understanding irreflexive concepts like 'taller than' and solving common-sense reasoning problems without being explicitly trained on those specific tasks.This is not the AI we were promised | The Royal Society
- Models are grown rather than built through training, with emergent capabilities that pop out unpredictably; researchers take bets on what models will be able to do rather than programming capabilities deliberately, making the process uninterpretable and unpredictable.AI Timelines and Human Psychology (with Sarah Hastings-Woodhouse)
- There is a meaningful distinction between having scaling continue at large parameter counts (moving from 1 trillion to 10 trillion to 10 quadrillion parameters) versus achieving surprising emergent capabilities where the algorithm does things that weren't anticipated purely through scale.Breakthrough potential of AI | Sam Altman | MIT 2023
- Narayan agrees with Acemoglu's assessment and adds that the AI industry's misleading intuition came from scaling laws: rapid progress from training larger models on more data led developers to believe that approach would continue and make domain expertise obsolete, but that scaling approach has run out because models are already trained on all available internet data and the remaining frontier is tacit knowledge that can't be learned passively.AI Snake Oil: What Artificial Intelligence Can Do, What It Can’t, and How to Tell the Difference