causal
Pretraining Compression Links to Intelligence
Pretraining improvements yield very general gains in intelligence because of a deep link between compression and intelligence, so a pretraining-automation loop makes the model broadly better.
causalpending
Speaker
Dwarkesh PatelEvidence Quote
“First, pretraining leads to very general improvements in the model's intelligence and capabilities. There is a deep explanation for why this is true, which has to do with the link between compression and intelligence.”
Source
Dwarkesh Patel Podcast - The avalanche that hit math is coming for AI development— Dwarkesh Patel PodcastCreated: 11/17/2026, 6:27:26 AM
Your notes
Loading notes...