causal

Generation-to-generation gains are mostly algorithmic

Improvements from one generation of these models to the next are partially driven by hardware and larger scale, but equally and perhaps even more so by major algorithmic improvements and changes in model architecture and training data mix that make the model better per FLOP applied.

causalpending

Speaker

Jeff Dean

Evidence Quote

“equally and perhaps even more so driven by major algorithmic improvements and major changes in the model architecture, the training data mix, and so on, that really makes the model better per flop”

Source

Jeff Dean & Noam Shazeer — 25 years at Google: from PageRank to AGI— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:30:41 AM

Your notes

Loading notes...