causal

Good Algorithms Failed Early For Lack Of Compute

Many correct deep-learning ideas existed early but failed for lack of compute to train them at scale; ResNets worked when published around 1988 but only at irrelevantly small scale and were forgotten until 2015, and BigGAN's success at 300 million images shows GANs scale, contradicting people who still claim GANs cannot scale past millions—demonstrating that abundant GPUs, not algorithms, are the common ingredient.

causalpending

Speaker

Gwern

Evidence Quote

“How can algorithms be so important when all these different generative architectures all work so well—as long as you have lots and lots of GPUs?" That's the common ingredient.”

Source

Gwern — Anonymous writer who predicted AI trajectory on $12K/year salary— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:28:51 AM

Your notes

Loading notes...