meta-learning
A machine learning approach that trains networks to learn how to learn, by optimizing initial weights so they can quickly learn new tasks from less data.
As defined in 343 | Tom Griffiths on The Laws of Thought
Where it comes up · 6
- Rich Sutton, The OaK Architecture: A Vision of SuperIntelligence from Experience - RLC 20252 claims
- Francois Chollet — Why the biggest AI models can't solve simple puzzles1 claim
- Fully autonomous robots are much closer than you think – Sergey Levine1 claim
- Dwarkesh Patel Podcast - Paul Christiano – We could be 12 months away from AIs that can overthrow the government1 claim
- Dwarkesh Patel Podcast - The avalanche that hit math is coming for AI development1 claim
- 343 | Tom Griffiths on The Laws of Thought1 claim
Most important claims · 7 of 7Search all claims for “meta-learning” →
- Meta-learning trains neural networks by manipulating their initial weights so they can learn from less data; using model-agnostic meta-learning, an outer loop adjusts the initial weights to improve performance across many learning problems (e.g., many languages) while an inner loop learns each individual problem, yielding a starting point that allows rapid learning from limited data and revealing the inductive biases human learners may need.343 | Tom Griffiths on The Laws of Thought
- Grokking—the transition from a memorized, overfit input-output mapping to a generalizing solution—is an instance of the minimum description length principle: the program that generalizes best is the shortest, achieved by progressively pruning and compressing the mapping, so it is memorization plus regularization rather than a separate mechanism.Francois Chollet — Why the biggest AI models can't solve simple puzzles
- Superintelligent systems will learn from many parallel deployment experiences at once, with learning algorithms that may be better than the ones evolution gave humans, so a few days of parallel experience could yield real-world expertise beyond the best human experts.Dwarkesh Patel Podcast - Paul Christiano – We could be 12 months away from AIs that can overthrow the government
- Simulation works for human pilots because they are intensely goal-directed toward flying the real airplane, whereas models trained on multi-domain data don't know they must solve a particular downstream task; the key to leveraging simulation is therefore building a foundation model good enough to have emergent ability to use auxiliary data, which itself requires the right objective grounded in real data.Fully autonomous robots are much closer than you think – Sergey Levine
- The OAK architecture requires solving reliable continual learning for deep neural networks, which currently fails due to catastrophic forgetting and loss of plasticity, with some solution approaches including continual backprop and meta-learning of new featuresRich Sutton, The OaK Architecture: A Vision of SuperIntelligence from Experience - RLC 2025
- Post-training has verifiable proxies: compute or data needed to RL on new environments, how quickly a model learns unseen games or tasks, and benchmarks like Anthropic's CoBench of around 500 real R&D problems, which can become RL environments that let models redo months of frontier progress.Dwarkesh Patel Podcast - The avalanche that hit math is coming for AI development
- Creating sub-problems for highly-ranked features is conceptually understood (blue checkmark) but waiting to be practically implemented once the problem of continual deep reinforcement learning with meta-learning is solvedRich Sutton, The OaK Architecture: A Vision of SuperIntelligence from Experience - RLC 2025