forecast

Long Context May Replace Continual Learning

Continual learning, defined as weights updating from deployment, may not be necessary if in-context learning becomes good enough across longer horizons, since six months of onboarding could in principle fit into an arbitrarily large context window.

forecastpending

Speaker

Dwarkesh Patel

Evidence Quote

“But what if you could just fit those six months into the context window? There's been tons of architectural innovations on the transformer which dramatically increase the length of context you can store.”

Source

Dwarkesh Patel Podcast - The next big breakthrough will be AIs learning on the job— Dwarkesh Patel Podcast
Created: 11/17/2026, 6:25:50 AM

Your notes

Loading notes...