forecast
Long Context May Replace Continual Learning
Continual learning, defined as weights updating from deployment, may not be necessary if in-context learning becomes good enough across longer horizons, since six months of onboarding could in principle fit into an arbitrarily large context window.
forecastpending
Speaker
Dwarkesh PatelEvidence Quote
“But what if you could just fit those six months into the context window? There's been tons of architectural innovations on the transformer which dramatically increase the length of context you can store.”
Source
Dwarkesh Patel Podcast - The next big breakthrough will be AIs learning on the job— Dwarkesh Patel PodcastCreated: 11/17/2026, 6:25:50 AM
Your notes
Loading notes...