factual

Data improvements gave three times model gains

For pretraining over the last six years, more than three times as much algorithmic progress has come from data improvements as from model improvements, according to research Patel did with Jerry Han.

factualpending

Speaker

Dwarkesh Patel

Evidence Quote

“more than three times as much algorithmic progress has come from data improvements as from model improvements.”

Source

Dwarkesh Patel Podcast - The avalanche that hit math is coming for AI development— Dwarkesh Patel Podcast
Created: 11/17/2026, 6:27:26 AM

Your notes

Loading notes...