causal

Algorithms follow hardware via cheap arithmetic

Deep learning took off largely because arithmetic became very cheap while moving data around became comparatively expensive; deep learning can be built from matrix multiplications that are N-cubed operations on only N-squared bytes of data communication, so algorithms have evolved to exploit this hardware economics.

causalpending

Speaker

Jeff Dean & Noam Shazeer

Evidence Quote

“arithmetic is very, very cheap, and moving data around is comparatively much more expensive. So pretty much all of deep learning has taken off roughly because of that. You can build it out of matrix multiplications that are N cubed operations and N squared bytes of data communication”

Source

Jeff Dean & Noam Shazeer — 25 years at Google: from PageRank to AGI— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:30:41 AM

Your notes

Loading notes...