test-time compute
Compute used at inference time (as opposed to training) to improve answers, e.g. via longer reasoning chains or sampling many candidates; central to reasoning models like o1 and o3.
As defined in Deepseek Dylan Patel Nathan Lambert
Other definitions
Allocating additional computation during inference so a model can reason longer before answering, improving output quality.
Nvidia v. Google, Scaling Laws, and the Economics of AI
Where it comes up · 10
- Capital, Compute & AI Scaling3 claims
- Dwarkesh Patel Podcast - Noam Brown – Agent swarms, alignment, & recursive self-improvement3 claims
- Nvidia v. Google, Scaling Laws, and the Economics of AI2 claims
- The AI Revolution Is Underhyped | Eric Schmidt | TED2 claims
- Week 1 of the Trump administration, AGI timelines, and DeepSeek2 claims
- AI's Trillion-Dollar Opportunity: Sequoia AI Ascent 2025 Keynote2 claims
- Inside the Trillion-Dollar AI Buildout1 claim
- Dwarkesh Patel Podcast - Eric Jang – Building AlphaGo from scratch1 claim
and 2 more
Most important claims · 10 of 18Search all claims for “test-time compute” →
- Despite physical improvements in hardware (the 'Grove's Law' pattern of exponentially faster chips), software engineers consume all gains and more through algorithmic complexity, so the historical rule 'Grove giveth, Gates taketh away' means that hardware speedups do not translate to proportional power savings—a pattern that will intensify as AI systems move from deep learning to reinforcement learning to test-time compute.The AI Revolution Is Underhyped | Eric Schmidt | TED
- The cost to achieve GPT-3-level intelligence has fallen roughly 1200x in about three years (from ~$60 per million tokens to a few cents), driven by architectural innovations, better data, better training techniques, and better hardware; this trend means we will have AGI-level intelligence before it permeates the economy due to inference cost constraints.Deepseek Dylan Patel Nathan Lambert
- The industry is shifting to a test-time compute paradigm in which the model explores multiple candidate solutions in parallel, uses a verifier to score and iterate on them, and scales intelligence along a time axis rather than a training-compute axis.Capital, Compute & AI Scaling
- Foundation models are getting better at penetrating the application layer through scaling laws, test-time compute, reasoning with tool use, and inter-agent communication, creating intense competition in the application layer.AI's Trillion-Dollar Opportunity: Sequoia AI Ascent 2025 Keynote
- Reasoning models that consume more test-time/inference-time compute to provide more intelligence are a compelling paradigm for math and coding, but for consumer products latency and intelligence-per-cost matter more — people won't wait half a minute for an answer when a good answer in half a second is a great tradeoff.Mark Zuckerberg — AI will write most Meta code in 18 months
- Test-time compute may exhaust useful search space quickly, verifier quality may not scale linearly with compute, and some tasks are limited by ambiguity or underlying knowledge rather than compute, so scaling test-time compute will not break all prior data limits.Capital, Compute & AI Scaling
- Pre-training scaling depended on the next generation of coherent chips, and since xAI could not get more than about 200,000 Hoppers coherent, pre-training progress would have stalled from mid-2024 until Gemini 3, but reasoning models let AI keep advancing without new hardware.Nvidia v. Google, Scaling Laws, and the Economics of AI
- Andy Jones's 'Scaling Scaling Laws with Board Games' showed you can trade off test-time compute and training compute—spending more compute on MCTS search yields performance equivalent to more training compute—anticipating LLM inference scaling, and additionally predicted how much compute is needed to solve a larger version of a board game.Dwarkesh Patel Podcast - Eric Jang – Building AlphaGo from scratch
- Cost per token has declined 99% over the last 12-18 months and cost curves will continue downward, though test-time compute may add some costs, the overall COGS trajectory is downward.AI's Trillion-Dollar Opportunity: Sequoia AI Ascent 2025 Keynote
- Reasoning-based test-time compute will likely be used in only a minority of queries, because users are impatient and speed is the most important consideration for application developers, so total compute needs will be bursty rather than constant.Capital, Compute & AI Scaling