causal
Architecture Not the Continual Learning Bottleneck
Architecture does not seem to be the fundamental bottleneck for continual learning, since many working ideas such as sparse attention and KV cache compaction already exist, so the loss function may be the real bottleneck.
causalpending
Speaker
Dwarkesh PatelEvidence Quote
“Perhaps the bottleneck is the loss function.”
Source
Dwarkesh Patel Podcast - The next big breakthrough will be AIs learning on the job— Dwarkesh Patel PodcastCreated: 11/17/2026, 6:25:50 AM
Your notes
Loading notes...