factual
Test-time compute overhang can add 4 OOMs
Current models only do a few hundred tokens of chain-of-thought before getting stuck, but unlocking 'System 2' coherent thinking over millions of tokens would represent roughly +4 OOMs of test-time compute on a single problem, which—per the AlphaGo train/test tradeoff—is comparable to a 3.5x-OOM bigger model, representing a large untapped overhang.
factualpending
Speaker
Leopold AschenbrennerEvidence Quote
“Right now, GPT-4 can do a few hundredtokenswithchain-of-thought... Suppose GPT-4 could think for millions of tokens. That's +4 OOMs on test time compute on one problem. It can't do it now. It gets stuck.”
Source
Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history— Dwarkesh Patel PodcastCreated: 6/13/2026, 3:30:02 AM
Your notes
Loading notes...