causal

Marginal learning cost undermines pretraining excuse

Even if pretraining on trillions of tokens can be explained as catching up with evolution, it does not explain why marginal capabilities require so much data, since an educated human does not need 100 different professors to learn a new programming language while pretrained AIs still do.

causalpending

Speaker

Dwarkesh Patel

Evidence Quote

“once you have been educated, you don’t need 100 different professors to learn a new programming language, but the AIs (even once pretrained) do.”

Source

Dwarkesh Patel Podcast - The data black hole at the center of AI— Dwarkesh Patel Podcast
Created: 11/17/2026, 6:11:50 AM

Your notes

Loading notes...