forecast
In-context-learning AGI claim requires huge context unlock
If, as Dario claims, in-context learning rather than continual learning suffices for AGI, then matching a month of working with an employee requires roughly hundred-million-token context lengths, which would demand a major ML-infra change to memory bandwidth (unless sparse attention substitutes)—a constraint Pope sees no good path to solving given HBM stagnation.
forecastpending
Speaker
Dwarkesh PatelEvidence Quote
“if you think that, then some ML infra thing would have to change to allow for a hundred million, like the memory bandwidth, to allow for a hundred-million-token context lengths.”
Source
Dwarkesh Patel Podcast - Reiner Pope – The math behind how LLMs are trained and served— Dwarkesh Patel PodcastCreated: 6/13/2026, 3:34:09 AM
Your notes
Loading notes...