factual

Catching up costs far less compute than being first

The compute required to be first to achieve something is always much larger than the compute needed to catch up, because followers can use tricks like distillation and good initializations as crutches; AlphaGo Zero used ~3E23 FLOPS as a huge outlier, but Jang reproduced a strong Go bot for roughly $10K of rented compute.

factualpending

Speaker

Eric Jang

Evidence Quote

“Once someone else has done it, you can use tricks like distillation. You can use all sorts of crutches to bootstrap your way to success.”

Source

Dwarkesh Patel Podcast - Eric Jang – Building AlphaGo from scratch— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:37:56 AM

Your notes

Loading notes...