causal
Algorithms follow hardware via cheap arithmetic
Deep learning took off largely because arithmetic became very cheap while moving data around became comparatively expensive; deep learning can be built from matrix multiplications that are N-cubed operations on only N-squared bytes of data communication, so algorithms have evolved to exploit this hardware economics.
causalpending
Speaker
Jeff Dean & Noam ShazeerEvidence Quote
“arithmetic is very, very cheap, and moving data around is comparatively much more expensive. So pretty much all of deep learning has taken off roughly because of that. You can build it out of matrix multiplications that are N cubed operations and N squared bytes of data communication”
Created: 6/13/2026, 3:30:41 AM
Your notes
Loading notes...