causal
Sample Efficiency and Continual Learning Are Linked
Sample efficiency and continual learning are deeply connected because on-the-job data is scarce and learning from it requires sample efficiency; in-context fast weights from attention scale poorly in memory, so architectures need an intermediate representation.
causalpending
Speaker
Dwarkesh PatelEvidence Quote
“sample efficiency and continual learning are actually deeply connected problems. Relatively little data is available to the model 'on-the-job'.”
Source
Dwarkesh Patel Podcast - The next big breakthrough will be AIs learning on the job— Dwarkesh Patel PodcastCreated: 11/17/2026, 6:25:50 AM
Your notes
Loading notes...