Training compute for the largest AI systems has grown by approximately 0.5 orders of magnitude per year for nearly a decade, a long-run trend that can be extrapolated forward to predict cluster sizes: GPT-4 at ~25,000 A100s ($500M) in 2022, scaling to 100,000 H100 equivalents ($billions) by 2024, 1 million H100s (10 GW, tens of billions) by 2026, 10 million H100s (100 GW, hundreds of billions) by 2028, and 100 million H100s (1 trillion dollars, 100 GW) by 2030 consuming over 20% of US electricity production.
forecastpending
Speaker
Leopold AschenbrennerEvidence Quote
“There's this long-run trend of almost a decade of training compute for the largest AI systems growing by about half an order of magnitude, 0.5 OOMs a year.”
Source
Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history— Dwarkesh PatelCreated: 8/12/2026, 6:39:27 PM
Your notes
Loading notes...