factual

Scaling Unlocks Emergent Abilities Nonlinearly

Since GPT-3 (2020), simply scaling models—more parameters, layers, and data—has been sufficient to produce breakthroughs, following scaling power laws for next-word prediction; in terms of actual capacities, performance improves nonlinearly, with abilities like solving math problems or commonsense reasoning emerging in transition-like phases past certain sizes, with notable connections to the physics notion of emergence.

factualpending

Speaker

Raphael Milliere

Evidence Quote

“there's this paper that was published by OpenAI in 2020 that finds this scaling power laws that look at how scaling the size of the model and the amount of data... leads to these improvements”

Source

230 | Raphaël Millière on How Artificial Intelligence Thinks— Sean Carroll's Mindscape
Created: 6/13/2026, 12:19:51 AM

Your notes

Loading notes...