sample inefficient
A property of current neural networks requiring enormous amounts of data to learn, far more than humans need for comparable tasks.
As defined in 336 | Anil Ananthaswamy on the Mathematics of Neural Nets and AI
Where it comes up · 1
Most important claims · 2 of 2Search all claims for “sample inefficient” →
- Scaling up large language models alone will not produce generalized intelligence because there is no mathematical guarantee a language model is accurate—it outputs a probability distribution with inbuilt stochasticity and only asymptotically approaches correctness—and because LLMs are extremely sample-inefficient; reaching transformative AI capable of generalizing beyond training data likely requires two or three further breakthroughs comparable to the attention paper.336 | Anil Ananthaswamy on the Mathematics of Neural Nets and AI
- A deep neural network given the same sparse data Kepler had (only tens of orbital positions) cannot reproduce Kepler's laws, because today's neural networks are extremely sample-inefficient and require far more data than humans, and Kepler also brought substantial prior knowledge of geometry and calculus.336 | Anil Ananthaswamy on the Mathematics of Neural Nets and AI