extrapolation
Generating outputs beyond the range of patterns in training data, often associated with genuine creativity.
As defined in AMA | August 2024
Other definitions
The cognitive habit of projecting imaginary negative futures from present circumstances; a key source of anxiety and worry that Stoics identify and teach people to interrupt.
31 of the absolute BEST lessons from Marcus Aurelius
Where it comes up · 9
- The AI Reset Is Here: Search, Jobs, and Everything Else w/ Anish Acharya, Dave Blundin, Salim Ismail2 claims
- 184 | Gary Marcus on Artificial Intelligence and Common Sense2 claims
- Kenneth Rogoff (Harvard University): "The dollar will always be there, but it will be less."2 claims
- 31 of the absolute BEST lessons from Marcus Aurelius1 claim
- The Most Important Thing - Origins and Inspirations | Howard Marks | Talks at Google1 claim
- Stephen Wolfram - Where the Computational Paradigm Leads (in Physics, Tech, AI, Biology, Math, ...)1 claim
- Jeremy Grantham on why this market will fall by 50% but nobody will warn you1 claim
- AMA | August 20241 claim
and 1 more
Most important claims · 10 of 12Search all claims for “extrapolation” →
- When deep learning makes an error, it is not behaving randomly but learning a systematic, lawful rule that simply differs from how humans see the universe—because rules are underdetermined by data (like '2,4,6,8' could continue many ways), and deep learning's output-node-driven computation favors interpolation over the rule-seeking extrapolation that humans are built for.184 | Gary Marcus on Artificial Intelligence and Common Sense
- Paul Samuelson and other leading 20th-century economists predicted in the early 1970s that the Soviet Union would economically catch up with the United States between 1987 and 1994, suggesting the ruble could become a significant global currency, based on simple extrapolation of Soviet growth trends without adequate data on the Soviet economy's structural problems.Kenneth Rogoff (Harvard University): "The dollar will always be there, but it will be less."
- Anxiety and worry come from extrapolation—imagining worst-case futures that haven't happened—and the Stoic practice is to stick with what is present and in your control, because you (not external events) are the common variable in all your anxieties.31 of the absolute BEST lessons from Marcus Aurelius
- The Soviet Union's collapse was shocking and unexpected, even though there were some warning signs, illustrating that confident extrapolation of the status quo ('nothing could ever change, things will always be the way they are') fails to anticipate fundamental transitions.Kenneth Rogoff (Harvard University): "The dollar will always be there, but it will be less."
- It is commonly claimed that large language models can only interpolate within their training data and not extrapolate or be creative like humans, but it is unclear whether humans actually extrapolate beyond their experience or also merely interpolate, raising the question of by what mechanism genuine extrapolation would occur.AMA | August 2024
- Extrapolation in economic forecasting works most of the time because the future usually resembles the recent past, but extrapolative forecasts don't make money because expected outcomes are already priced into securities.The Most Important Thing - Origins and Inspirations | Howard Marks | Talks at Google
- Neural networks fail at extrapolation: training a neural net to reproduce a sine curve in one region and then asking it to extrapolate beyond that region produces terrible results, even when the network size is increased.Stephen Wolfram - Where the Computational Paradigm Leads (in Physics, Tech, AI, Biology, Math, ...)
- Large language models are effective at interpolating (inferring patterns between known data points) but not at extrapolating (predicting beyond the training data), which means even if trained on physics first principles, they will fill in blind spots based on assumptions rather than true reasoning.The AI Reset Is Here: Search, Jobs, and Everything Else w/ Anish Acharya, Dave Blundin, Salim Ismail
- The goal for AI should not be to merely equal human intelligence—which Kahneman called a low bar—but to exceed it, because if we are going to trust AI and put it in charge of consequential decisions, it must be free of confirmation bias and racist stereotypes and able to extrapolate to the world we want, making it better than all people.184 | Gary Marcus on Artificial Intelligence and Common Sense
- Extrapolation is the opiate of the people—straight-line projection of past trends into the future is the primary error in forecasting and was made worse by spreadsheet software; real forecasting requires understanding a range of possible outcomes and identifying which are plausible given historical patterns, not projecting the most recent trend forward.10 year anniversary presentation -- Russell Napier on lessons from financial history