causal
Good Algorithms Failed Early For Lack Of Compute
Many correct deep-learning ideas existed early but failed for lack of compute to train them at scale; ResNets worked when published around 1988 but only at irrelevantly small scale and were forgotten until 2015, and BigGAN's success at 300 million images shows GANs scale, contradicting people who still claim GANs cannot scale past millions—demonstrating that abundant GPUs, not algorithms, are the common ingredient.
causalpending
Speaker
GwernEvidence Quote
“How can algorithms be so important when all these different generative architectures all work so well—as long as you have lots and lots of GPUs?" That's the common ingredient.”
Source
Gwern — Anonymous writer who predicted AI trajectory on $12K/year salary— Dwarkesh Patel PodcastCreated: 6/13/2026, 3:28:51 AM
Your notes
Loading notes...