causal
Over-trained specialists generalize worse
A student who trains 10,000 hours to win competitive programming tends to do worse in their career than one who reaches the top with 100 hours, and current models resemble the over-trained first student, which intuitively explains why heavily-trained models fail to generalize beyond their training domain.
causalpending
Speaker
Ilya SutskeverEvidence Quote
“The models are much more like the first student, but even more.”
Source
Ilya Sutskever – We're moving from the age of scaling to the age of research— Dwarkesh Patel PodcastCreated: 6/13/2026, 3:24:31 AM
Your notes
Loading notes...