forecast

In-context-learning AGI claim requires huge context unlock

If, as Dario claims, in-context learning rather than continual learning suffices for AGI, then matching a month of working with an employee requires roughly hundred-million-token context lengths, which would demand a major ML-infra change to memory bandwidth (unless sparse attention substitutes)—a constraint Pope sees no good path to solving given HBM stagnation.

forecastpending

Speaker

Dwarkesh Patel

Evidence Quote

“if you think that, then some ML infra thing would have to change to allow for a hundred million, like the memory bandwidth, to allow for a hundred-million-token context lengths.”

Source

Dwarkesh Patel Podcast - Reiner Pope – The math behind how LLMs are trained and served— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:34:09 AM

Your notes

Loading notes...