Loss function
In machine learning, a function quantifying the error of a model's predictions, which training algorithms minimize.
As defined in Diane Coyle on Cogs, Monsters, and Better Economics
Other definitions
A mathematical function that measures how far a model's predictions are from desired outputs; gradient descent minimizes the loss function
Live: Eliezer Yudkowsky - Is Artificial General Intelligence too Dangerous to Build?
Where it comes up · 18
- Adam Marblestone – AI is missing something fundamental about the brain4 claims
- Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality4 claims
- LeWorldModel: Stable End-to-End JEPA World Models From Pixels3 claims
- Dario Amodei (Anthropic CEO) — The hidden pattern behind every AI breakthrough2 claims
- 324 - Debating The Future Of Ai2 claims
- Live: Eliezer Yudkowsky - Is Artificial General Intelligence too Dangerous to Build?1 claim
- Demis Hassabis — Scaling, superhuman AIs, AlphaZero atop LLMs, AlphaFold1 claim
- Fully autonomous robots are much closer than you think – Sergey Levine1 claim
and 10 more
Most important claims · 10 of 28Search all claims for “Loss function” →
- Machine learning systems increasingly used in the private and public sectors to make consequential decisions about people's lives are literally utility maximizers—set up to maximize objective functions or minimize loss functions—chosen by computer scientists, despite all the unresolved social choice problems economists know.Diane Coyle on Cogs, Monsters, and Better Economics
- Ronald Howard's exam scheme, where students stated their probability of an answer being correct and a 100% confident wrong answer yielded a negative-infinity grade, vividly teaches that there is no such thing as a sure bet and that overconfident all-in commitments can produce irrecoverable game-over outcomes.Luca Dellanna on Risk, Ruin, and Ergodicity
- The historical problem with training world models through prediction is 'representation collapse'—where an AI can achieve perfect prediction accuracy by mapping all inputs to the same blank representation (e.g., zero) without learning any actual physics, because the loss function only penalizes prediction error, not learning quality.LeWorldModel: Stable End-to-End JEPA World Models From Pixels
- When you take an optimizer like natural selection or gradient descent and optimize a complex system hard enough for X that it becomes a general intelligence doing X, the resulting intelligence has no explicit goal of X: natural selection hill-climbed humans for inclusive genetic fitness, yet humans have hundreds of small goals and none is fitness maximization, and the same instability holds for any current method of producing general intelligence.116 - Ai Racing Toward The Brink
- Shane Legg's 2009 thesis proposed that a test for AI is whether you can compress Wikipedia, which is essentially the loss function used to train modern LLMs, and Hassabis's own 2016 paper comparing neuroscience and AI identified attention as a needed capability before transformers existed.Demis Hassabis — Scaling, superhuman AIs, AlphaZero atop LLMs, AlphaFold
- The backpropagation and gradient descent math is identical whether a network has a handful of parameters or close to a trillion (as in GPT-4, O1, O3, Claude); the loss landscape simply becomes a function of that many parameters, and the core algorithm from 1986 still holds despite modern engineering innovations like stochastic gradient descent and momentum.336 | Anil Ananthaswamy on the Mathematics of Neural Nets and AI
- Taking a giant inscrutable array of floating point numbers and differentiating it against a loss function to drive the loss lower is analogous to natural selection rather than to writing code you understand: you blindly explore a space of possibilities and make things that solve the problem without understanding how, which is Step One toward creating things with strange inscrutable desires.Eliezer Yudkowsky on the Dangers of AI
- LayWM (Low-Rank World Model) is co-authored by a team including AI pioneer Yann LeCun and learns intuitive physics directly from raw video pixels using a novel two-term loss function instead of the seven-term loss function required by older models like PLDM.LeWorldModel: Stable End-to-End JEPA World Models From Pixels
- Simulation works for human pilots because they are intensely goal-directed toward flying the real airplane, whereas models trained on multi-domain data don't know they must solve a particular downstream task; the key to leveraging simulation is therefore building a foundation model good enough to have emergent ability to use auxiliary data, which itself requires the right objective grounded in real data.Fully autonomous robots are much closer than you think – Sergey Levine
- Both AIs and humans undergo phase shifts from imitative to structural understanding: a human first imitates jazz sounds, then suddenly grasps the underlying structure and can improvise; in training this is called grokking, where loss slowly improves then suddenly the system develops a fundamentally different, more efficient way of doing things—and the worry is that such shifts break our current methods of keeping the system in check, predicting it, and ensuring it does what we want.Zvi Mowshowitz on AI and the Dial of Progress