interpolative memory
A model that stores patterns and knowledge as a fitted curve and answers by interpolating between memorized points, rather than reasoning anew.
As defined in Francois Chollet — Why the biggest AI models can't solve simple puzzles
Where it comes up · 1
Most important claims · 3 of 3Search all claims for “interpolative memory” →
- The way you measure performance is not a technical detail but the central issue: scaling laws relate compute to benchmark performance, and because nearly all LLM benchmarks are memorization-based (solvable by memorizing a finite set of reasoning patterns), scaling increases skill and usefulness but not intelligence—conflating skill with intelligence is the fundamental confusion.Francois Chollet — Why the biggest AI models can't solve simple puzzles
- LLMs are big interpolative memories—parametric curves fitted to a data distribution—that maintain a bank of memorized solution programs; when given a puzzle they fetch and apply the appropriate stored program, which looks like reasoning but is program fetching, not on-the-fly program synthesis.Francois Chollet — Why the biggest AI models can't solve simple puzzles
- Memory and intelligence are separate components: scaling up the interpolative memory is the easy part (just a big curve needing more data), while the hard, non-obvious part is the architecture of intelligence (System 2); we currently have the memory but not the intelligence, though an intelligence not hooked up to extensive memory would also be of limited use.Francois Chollet — Why the biggest AI models can't solve simple puzzles