verifiable domain
A field where outputs can be checked automatically against a clear criterion, such as passing unit tests or proving a theorem, enabling many rounds of training practice.
As defined in Dwarkesh Patel Podcast - The avalanche that hit math is coming for AI development
Other definitions
A task where outputs can be checked quickly and automatically, such as code passing tests or ads producing clicks, which makes AI training easier.
Ben Thompson on Big Tech, China, and the AI Boom Running Out of Money - [Invest Like the Best, EP.487]
Where it comes up · 1
Most important claims · 6 of 6Search all claims for “verifiable domain” →
- In every domain where answers can be checked and AIs can practice thousands of rounds, such as competitive programming and well-scoped open math problems, models have gone from worse than most humans to better than the best humans within a few years.Dwarkesh Patel Podcast - The avalanche that hit math is coming for AI development
- In verifiable domains like mathematics, AIs are doing search-and-match discovery so effectively that hundreds of important results, which top mathematicians had spent decades trying to resolve, are being produced as side quests to hype future model releases.Dwarkesh Patel Podcast - The avalanche that hit math is coming for AI development
- Math progress has skewed disproportionately toward verification-laden parts like finding counterexamples rather than creating new theories, and Toby Ord argues there is little evidence AI can ask powerful questions or find profound new fields of mathematical inquiry.Dwarkesh Patel Podcast - The avalanche that hit math is coming for AI development
- A similar jump starting in 2027 or 2028 would produce a mind vastly superhuman in slightly verifiable domains, better than Nvidia at designing next-generation AI accelerators, better than Citadel or Jane Street at trading if given equal market access, and better than all humanity at finding a room-temperature superconductor given lab access.Dwarkesh Patel Podcast - The avalanche that hit math is coming for AI development
- AI R&D has three broad buckets, pretraining, post-training, and inference efficiency, each with strong verifiable proxies, and since together they comprise all algorithmic progress, AI progress is generally easy to train against.Dwarkesh Patel Podcast - The avalanche that hit math is coming for AI development
- Some parts of AI research are less verifiable than math, which is why OpenAI has not fired all its researchers, but AI research is verifiable enough that AIs will soon match the best humans at it.Dwarkesh Patel Podcast - The avalanche that hit math is coming for AI development