Scaling law
A mathematical relationship in which a quantity varies as a power of another, often signaling scale-free or self-similar structure.
As defined in ms_260
Other definitions
A quantitative relationship where a property or quantity scales with another variable (usually size) according to a power law, with a consistent exponent that applies across diverse systems
Searching for Simplicity & Unity | Geoffrey West | Talks at Google
Where it comes up · 14
- 226 | Johanna Hoffman on Speculative Futures of Cities8 claims
- Universal Mathematics: All Life on Earth Is Bound by One Spooky Algorithm | Geoffrey West5 claims
- Searching for Simplicity & Unity | Geoffrey West | Talks at Google4 claims
- Ai Sota #20264 claims
- AI's Trillion-Dollar Opportunity: Sequoia AI Ascent 2025 Keynote3 claims
- John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI2 claims
- Your AI Stack Isn't Ready for Claude Mythos2 claims
- Gavin Baker - The Cyclone Under the Surface - [Invest Like the Best, EP. 260]2 claims
and 6 more
Most important claims · 10 of 38Search all claims for “Scaling law” →
- Like biological organisms, cities follow power-law scaling, but infrastructure quantities such as gas stations, roads, electrical lines, and water lines scale with an exponent of about 0.85, meaning doubling a city's size requires only about 85% more infrastructure—a systematic economy of scale that holds across countries worldwide.226 | Johanna Hoffman on Speculative Futures of Cities
- RLVR follows a scaling paradigm where log-increasing training compute yields a linear increase in evaluation scores (reproduced by OpenAI's o1 and DeepSeek), but there is no such scaling law for RLHF—in fact the seminal RLHF scaling paper is about reward model over-optimization—which is field-defining because to do the best RLVR you need the extra 10-100x compute but for RLHF you don't.Ai Sota #2026
- The existence of simple, universal scaling laws in biology is surprising because the naive view of natural selection—random processes in environments producing historically contingent organisms—predicts that plotting metabolic rate against size would scatter points all over the graph, yet instead they fall on a tight power law, suggesting something transcends evolutionary contingency.226 | Johanna Hoffman on Speculative Futures of Cities
- Metabolic rate (the daily energy or food an organism needs) scales with the three-quarter power of body mass across mammals from the smallest to the blue whale, a remarkably simple and regular power law (Kleiber's law) that means larger organisms need more total energy but systematically less per gram.226 | Johanna Hoffman on Speculative Futures of Cities
- Across mammals, heart rate and lifespan scale inversely with size in a way that approximately cancels, so the total number of heartbeats in a lifetime is roughly constant (about 1.5 billion in nature, ~3 billion for modern humans) regardless of whether the animal is a mouse or a blue whale.226 | Johanna Hoffman on Speculative Futures of Cities
- The same quarter-power family of exponents recurs across dozens of distinct physiological and life-history traits—heart rate scales as the minus one-quarter power of mass, aorta radius as the three-eighths power, lifespan, maturation time, and growth rate all show related exponents—so the number four mysteriously permeates and constrains the diversity of life.226 | Johanna Hoffman on Speculative Futures of Cities
- Socioeconomic quantities in cities—wages, construction, patents, innovation, crime, and disease cases—all increase superlinearly with size at an exponent of about 1.15, meaning every doubling of population yields roughly 15% more per capita, driven by the positive feedback loop of social interaction that cities facilitate; this is the inverse of biology where larger means less per capita.226 | Johanna Hoffman on Speculative Futures of Cities
- Cell phone call data from millions of people across UK and Portugal shows that human social interactions between people scale with city size at the same 1.15 exponent as socioeconomic metrics, confirming that social network mathematics underlies scaling lawsSearching for Simplicity & Unity | Geoffrey West | Talks at Google
- Foundation models are getting better at penetrating the application layer through scaling laws, test-time compute, reasoning with tool use, and inter-agent communication, creating intense competition in the application layer.AI's Trillion-Dollar Opportunity: Sequoia AI Ascent 2025 Keynote
- If scaling laws are fundamental in deep learning, the Bitter Lesson will always apply because compute becomes more abundant—but even with abundant compute, the methods with a steeper scaling-law slope or better offset (getting 100x out of the same compute) will win, so leverage on compute still matters.Ai Sota #2026