factual
Catching up costs far less compute than being first
The compute required to be first to achieve something is always much larger than the compute needed to catch up, because followers can use tricks like distillation and good initializations as crutches; AlphaGo Zero used ~3E23 FLOPS as a huge outlier, but Jang reproduced a strong Go bot for roughly $10K of rented compute.
factualpending
Speaker
Eric JangEvidence Quote
“Once someone else has done it, you can use tricks like distillation. You can use all sorts of crutches to bootstrap your way to success.”
Created: 6/13/2026, 3:37:56 AM
Your notes
Loading notes...