Dwarkesh Patel Podcast - The avalanche that hit math is coming for AI development
What this covers
The episode is a single-narrator argument from Dwarkesh Patel that starts with mathematics. AI went from failing four-digit addition in 2023 to an IMO gold medal in 2025, then to open Erdős problems and, by his account, a Millennium Prize problem solved in 88 hours. He treats this as a template for what happens wherever answers can be checked, and asks how much of AI research is that kind of verifiable domain. His evidence includes METR's finding, after three months inside Anthropic, that the lab's rate of progress on well-scoped tasks has doubled. He also points to held-out loss as a direct measure of pretraining progress, and to Magic's claim of a base model for roughly $500,000.
The middle of the piece takes up the objections. Patel discusses reinforcement learning in datacenters versus real-world work like running businesses or winning elections, and the lack of continual learning, where every session is a first day on the job. He bets that longer in-context sessions may be enough. He also argues about research taste: the transformer was a combination of known ideas, evolution produced a general learner from a crude verifier, and Toby Ord doubts AI can yet pose powerful new questions.
The closing stretch is quantitative. Ideas are getting harder to find, with R&D compute quadrupling and staff doubling each year just to hold pace. Automated researchers would run about 50 times more work per calendar week than a human, though compute limits bite hard. Global compute grows about 3x a year, so the endpoint arrives over years rather than months. DeepSeek is cited as finding that data and environment quality matter more than RL algorithm changes.
Patel argues that AI research is highly verifiable, so AI is already accelerating its own progress, and automated AI researchers could match the best human researchers by around 2028 (possibly 2027). That would trigger either a software-only singularity or a 'spark' that compresses several years of progress into one year.
- METR and Anthropic/OpenAI usage data show AI already speeds up frontier AI R&D.
- Math progress shows AIs excel at verifiable, well-scoped search-and-match discovery, and AI research has similar verifiable proxies.
- Bottlenecks like continual learning, data, and experiments slow but do not cancel the takeoff.
This asset isn't compiled yet
You're seeing its claims, ranked. Compile it to build the argument threads, weight them, and check each claim against your library — the full view.
Automated researchers could develop a much better science of learning from small experiments, extending scaling laws and discovering phenomena like muP, which predicts optimal hyperparameters for larger runs from models 100x smaller, and GPT-4's loss was predicted from models trained with 10,000x less compute.
“the GPT-4 paper says that they were able to predict that model’s final loss using models trained with 10,000x less compute.”
Even without AI help, global AI compute has been growing roughly 3x per year, and spark-era AIs would help build more chips, datacenters, and robots, so reaching superintelligence takes years rather than months, but the endpoint is similar.
“Even without help from AIs, global AI compute has been growing roughly 3x a year.”
AI companies have had to quadruple their R&D compute and double their workforce every single year just to maintain the same rate of progress, showing that ideas are getting harder to find.
“AI companies have to quadruple their R&D compute and double their workforce every single year just to maintain the same rate of progress.”
Evolution optimized humans using a very crude verifier, whether genes were passed on, with ancestral tasks like hunting and tribal standing, yet it produced a general-purpose learner; deliberate AI training with better-matched tasks should therefore transfer even further.
“Our general-purpose mental architecture was developed by evolution, which is an optimization process with a really crude verifier: Did you pass your genes on or not?”
DeepSeek reports that changes to the basic RL algorithm have not mattered nearly as much as systematic improvements in the scale, diversity, and verifiability of synthesized data and environments, which account for essentially all observed gains.
“systematic improvements in the scale, diversity, and verifiability of synthesized data and environments account for essentially all of the observed gains.”
Proxies made before a paradigm leap still capture large step changes because AI learning algorithms are general: the transformer was visible on the translation benchmark it was invented to beat, and paradigm-agnostic outer-loop proxies like unseen-game mastery or pre-1905 physics discovery remain available.
“The transformer was visible on (in fact was invented to beat) the existing translation benchmark”
METR, after three months embedded inside Anthropic, concluded that Anthropic's overall rate of progress on well-scoped tasks is 2x what it was a year ago, and that a majority of this acceleration is likely attributable to Anthropic's extensive use of AI to do R&D.
“Anthropic’s overall rate of progress on well-scoped tasks is 2x what it was a year ago, and that a “majority of this acceleration today is likely attributable to Anthropic’s extensive use of AI to do R&D.””
Today's AIs already work at about 10 times human speed, and because automated researchers would work around the clock, each one would cram roughly 50 times more work into every calendar week than a human researcher, though compute bottlenecks would bring steep diminishing returns.
“each individual automated researcher will be cramming in roughly 50x more work every calendar week than a human researcher can do.”
The transformer was a natural combination of ideas, such as attention and residual networks, that had independently been effective, so none of its core pieces were surprising to researchers in the field.
“the transformer architecture was a natural combination of many ideas (attention, residual networks) which had independently been effective”
Current AI is trained with RL on tasks that are easy to grade within a datacenter, but most of the world, such as building businesses, winning court cases, trading, and winning elections, requires real-world interaction that cannot be recreated inside a datacenter.
“The way we’re currently making AIs smarter is by RLing them on tasks that are easy to automatically grade from within a datacenter”
Sample efficiency and online learning are two sides of the same problem of learning from real-world data quickly, a deep open problem in machine learning that is AI's equivalent of a Millennium Prize problem, but every kind of sample efficiency is measurable and therefore verifiable.
“Sample efficiency and online learning are really two sides of the same problem: how to learn from real-world data quickly and efficiently.”
Pretraining progress is extremely easy to verify because loss on a held-out test set is a direct measure, so GPT-7 could be RL-trained to minimize compute needed to reach a fixed loss, and swarms could search for the lowest perplexity.
“Loss on a held-out test set, which is the basis of pretraining progress, is extremely easy to verify.”
Because models generalize far beyond what they were trained on, models trained on less ambitious verifiable math problems can solve problems no human had cracked, so the same generalization should make models excellent at unverifiable parts of AI research like research taste.
“models generalize far beyond what they were directly trained on”
Before the 2017 transformer paper, attention, residual networks and other ideas were already known, and the transformer was only obviously better once half a dozen additional tweaks were combined, which required research taste and many small experiments.
“One reason was that this new architecture was only obviously better than the state of the art once you connected half a dozen additional tweaks and innovations.”
AI math capability went from failing at four-digit addition in 2023, to barely solving high school competition problems in 2024, to winning an IMO gold medal in 2025, to solving open Erdős problems, and about a month before recording solving a Millennium Prize problem in 88 hours.
““Okay, it fuckingsolved a Millennium Prize problemin 88 hours.””
Pretraining improvements yield very general gains in intelligence because of a deep link between compression and intelligence, so a pretraining-automation loop makes the model broadly better.
“There is a deep explanation for why this is true, which has to do with the link between compression and intelligence”
Scaling in-context learning so sessions last many months may be enough for a single AI instance to learn on the job effectively, without an architectural breakthrough that updates weights at the end of each session.
“scaling up in-context learning, so that you have sessions that are effectively many months long, will probably be enough to let a single instance learn on the job effectively.”
Patel does not think every real-world answer can be baked into model weights; rather, scaling up RL training will give AIs the abilities to ramp quickly in new contexts, even in hard-to-verify domains like running businesses, court cases, trading, or elections.
“my claim is that scaling up RL training will give AIs the abilities necessary to ramp quickly in new contexts, even in hard-to-verify domains”
In every domain where answers can be checked and AIs can practice thousands of rounds, such as competitive programming and well-scoped open math problems, models have gone from worse than most humans to better than the best humans within a few years.
“In every domain where we can check answers and give AIs thousands of rounds of practice ... they've gone from worse than most humans to better than the best humans within a few years.”
In verifiable domains like mathematics, AIs are doing search-and-match discovery so effectively that hundreds of important results, which top mathematicians had spent decades trying to resolve, are being produced as side quests to hype future model releases.
“hundreds of important results (which the top mathematicians and computer scientists in the world had spent decades trying to resolve) are being churned out as side quests”
Current AIs lack the ability to improve from on-the-job experience: every new session is effectively a first day on the job, and each session retains only a couple of hours of information, so they do not accumulate weeks or months of feedback.
“Every new session is their first day on the job. And in each session, they can only retain a couple of hours’ worth of information”
The AI lab Magic announced it pretrained a model matching leading open-source base models such as DeepSeek V4 Pro for just half the compute used to train the original GPT-3, roughly 500,000 dollars today, and Patel thinks orders-of-magnitude further improvements are plausible.
“they pretrained a model that matches the leading open-source base models today, like DeepSeek V4 Pro, for just half of the compute used to train the original GPT-3”
OpenAI said GPT-5.2, which is dumber and less efficient than today's models, completed GDPval tasks 11 times faster than human experts, and a frontier model outputs about 60 tokens per second, roughly 13 times faster than a human talks.
“OpenAI said that GPT-5.2, which is dumber and less efficient than today’s models, completed the tasks on its GDPval benchmark 11 times faster than human experts.”
Math progress has skewed disproportionately toward verification-laden parts like finding counterexamples rather than creating new theories, and Toby Ord argues there is little evidence AI can ask powerful questions or find profound new fields of mathematical inquiry.
“It’s true that progress has skewed disproportionately towards the verification-laden parts of math, like finding counterexamples (rather than coming up with new theories, say).”
As of August, Claude 'leads' more than 26% of Anthropic's AI R&D work, up from under 1% in February; 'leads' means completing most of a task end-to-end from a high-level prompt with a human supervising, while fully autonomous work remains at 0%.
“as of August, Claude “leads” more than 26% of Anthropic’s AI R&D work—up from under 1% as recently as February.”
OpenAI reported that the heaviest 10% of its researchers' AI users went from using around $1,000 a day of internal coding agents at the start of summer to over $7,000 a day by the end of August.
“the heaviest 10% of AI users among OpenAI’s researchers went from using around $1,000 a day of internal coding agents at the beginning of the summer to over $7,000 a day by the end of August.”
All the progress made in deep learning over the last decade and a half has not required much new theory, meaning empirical recipe-finding has driven the field.
“all the progress we've made in deep learning over the last decade and a half has not required much new theory anyway”
Even at one ten-thousandth of the RL training scale expected during the intelligence explosion, current models already generalize: Claude Code or Codex can orient to an unfamiliar project, figure out how it fits together, and tackle an ambitious assignment.
“even at one ten-thousandth of the scale of RL training that will be happening during the intelligence explosion, we’re already seeing lots of generalization.”
Research taste is itself verifiable: an RL environment can ask GPT-7 to predict whether an algorithmic improvement made at 100M parameters will hold at 10B, a taste-bench that is costly but feasible on frontier compute.
“you can make an RL environment where GPT-7 has to predict whether an algorithmic improvement made at the 100M-parameter scale will hold at the 10B scale.”
A similar jump starting in 2027 or 2028 would produce a mind vastly superhuman in slightly verifiable domains, better than Nvidia at designing next-generation AI accelerators, better than Citadel or Jane Street at trading if given equal market access, and better than all humanity at finding a room-temperature superconductor given lab access.
“you’d probably get a mind that would be vastly superhuman in domains that are even slightly verifiable.”
Well-scoped problem solving matters even more for machine learning research than for mathematics, because mathematics values new theory for its own sake while AI labs care mainly about whether an algorithmic recipe empirically improves training efficiency or model smartness.
“But in AI research, the priorities are exactly reversed. AI labs mainly care about how any new algorithmic recipe empirically makes training more efficient or elicits smarter models.”
Automated researchers can trade labor for compute: where human researchers waste compute to save scarce researcher time, agents can run many more small ablations, read every rollout, and rebuild broken environments, so experiment compute is used far more effectively than today.
“Immediately, they can trade labor for compute. To save valuable researcher time, experiments today are, and should be, very wasteful with compute.”
AI research is closer to formalizing a theorem than writing a novel, because it contains many independent modular subtasks with verifiable proxies, so a swarm of agents can try a thousand approaches per part of the pipeline and speed progress a lot.
“AI research is closer to formalizing a theorem than to writing a novel, because there is a lot of independent modular work to do, and there are verifiable proxies for progress.”
Automating AI research will compress so much progress into a single year that, soon thereafter (by 2030, plausibly as early as next year), AIs will be at least as good as the best human experts in every field.
“automating AI research will compress so much progress into a single year that, soon thereafter (so by 2030, but plausibly as early as next year), we’ll have AIs that are at least as good as the best human experts in every field.”
A SlothAI with the same compute but researchers working one hour a week would make algorithmic progress at least five times slower, and since automated companies are to OpenAI as OpenAI is to SlothAI, human thinking speed is a biological accident that sets no special limit on useful thinking speed.
“It seems to me that, intuitively, SlothAI would end up making algorithmic progress at least five times slower.”
If ideas get harder to find faster than recursive self-improvement makes AIs better at research, the result is a 'spark' of roughly three to six years of AI progress in a single year, followed by a slowdown until more compute is built.
“we’ll end up with what I’d call a “spark.” We’d get something like three to six years of AI progress in a single year.”
During the intelligence explosion, automated engineers will build tens of thousands of times more RL training environments than exist today, including long-running simulated economies where AIs cofound companies through centuries of simulated corporate competition.
“During the intelligence explosion, automated engineers will build tens of thousands of times more RL training environments than we have currently.”
An AI shadowing a CEO could read every email, Slack message and document on its first morning, and parallel copies could interview every employee and major client, doing a human's months of work by the end of day one, so AIs could reach top human performance in most domains within months of deployment, though feedback on quarterly results requires serial time.
“By the end of the first day, parallel copies of it could have interviewed every employee and major client, and cumulatively done the amount of work it would have taken a single human months to do”
AIs can train on prediction tasks built from history, such as deriving general relativity from a corpus ending before 1700 or predicting the Cuban Missile Crisis from newspapers through 1962, developing scientific and political judgment beyond the shrewdest human forecasters.
“Can an AI figure out general relativity given a training corpus that includes no information beyond 1700”
For pretraining over the last six years, more than three times as much algorithmic progress has come from data improvements as from model improvements, according to research Patel did with Jerry Han.
“we found that for pretraining, over the last six years, more than three times as much algorithmic progress has come from data improvements as from model improvements.”
Automating pretraining would radically speed AI progress for three reasons: pretraining gives general intelligence gains, smaller models trained faster can match larger ones and support more researchers, and faster small-model training tightens the RSI loop because giant runs become obsolete before finishing.
“By automating pretraining alone, you’d radically speed up AI progress.”
Getting good at a complex task requires two things: domain-specific knowledge and a more general set of skills like seeking information, making plans, persisting through mistakes, tracking what you've learned, and collaborating with others.
“Getting really good at any complex task takes two things. One is domain-specific knowledge. The other is a more general set of skills”
Today's experiment practices reflect that researcher time is extremely scarce relative to compute, since top researchers earn hundreds of millions a year, so current compute use tells us little about how efficiently labs could use compute if compute were the key constraint.
“researcher time is extremely scarce relative to compute—top AI researchers individually get paid hundreds of millions of dollars a year.”
As pretraining improves, smaller models can match larger ones while training faster and cheaper, which speeds experiments and lets labs train models at a scale where thousands of experiments run daily, so small-scale results need not be bet on bigger models.
“because smaller models train faster, this tightens the RSI loop itself.”
If the rate at which AIs get smarter outpaces the rate at which ideas get harder to find, a software-only singularity follows, where without more compute the AI swarm produces a superintelligence scraping against the physical limits of current chips within months.
“we’d get something called the “software-only singularity.” In this scenario, even without increasing compute, the AI swarm rapidly makes smarter and smarter versions of itself”
AI research resembles the mathematical progress AIs are good at: unlocking large amounts of low-hanging fruit through persistence and a superhuman ability to connect ideas from many adjacent domains.
“AI research is actually surprisingly similar to the type of mathematical progress that current AIs are disproportionately good at: unlocking a ton of low-hanging fruit by applying a combination of incredible persistence and a superhuman capacity to connect ideas from lots of adjacent domains.”
Napoleon at 27 and Rockefeller at 25 ramped up in domains they were new to, needing only a couple years of experience rather than decades, so AIs should be able to replicate or exceed that learning speed.
“Rockefeller and Napoleon needed experience, yes, but only a couple of years’ worth, not decades.”
Choosing which objective to optimize is part of research taste, but paradigm-agnostic outer-loop proxies such as Millennium Prize solves, time to rediscover general relativity from pre-1905 data, and speed of mastering unseen games can verify progress across paradigms.
“It's not that difficult to come up with paradigm-agnostic outer loop proxies”
Frontier labs quadruple compute yearly, so by the end of 2027 OpenAI could give thousands of agents enough compute each to run a GPT-3-sized training run daily, and the trade of labor for compute means experiment bottlenecks are weaker than they look.
“If this trend continues, then by the end of 2027, OpenAI could give thousands of agents each enough compute to run a GPT-3-sized training run every single day.”
Because online learning has clean verifiable proxies, models a year or two smarter than current ones might crack continual learning even without persistent memory, and it was not obvious in advance that continual learning would be harder than Navier-Stokes.
“But are you so confident that it’s that much harder that non-online-learning models a year or two smarter than the ones currently solving Millennium Prize problems won’t be able to crack it”
Around 80% of the algorithmic progress at a given frontier lab is driven by its top 50 researchers, so automated researchers matching the very best humans would yield a large additional speedup.
“My sense is that around 80% of the algorithmic progress at a given frontier lab is driven by its top 50 researchers.”
Automated researchers will radically improve learning algorithms through online learning and sample efficiency breakthroughs, and future AIs could be far beyond humans in learning efficiency, since humans learn from data far more efficiently than LLMs do.
“In the same way that humans are much more efficient and effective at learning from data than current LLMs are, future AIs could be far beyond humans in learning efficiency.”
Because matching dozens of idea combinations is what AIs are disproportionately skilled at, a model like Claude 7 could invent something as large as the transformer if Chris Olah asked it to hard enough to improve attention.
“is the kind of thing that Claude 7 could invent if Chris Olah asked it to try hard to get this attention thing to work better.”
Current AI systems have solved a Millennium Prize problem without the ability to ask powerful questions or find profound new fields of mathematical inquiry, so well-scoped problem solving can be a major capability independent of theory-generation.
“Current AI systems have solved a Millennium Prize problem, all without the ability to ask powerful questions”
Post-training has verifiable proxies: compute or data needed to RL on new environments, how quickly a model learns unseen games or tasks, and benchmarks like Anthropic's CoBench of around 500 real R&D problems, which can become RL environments that let models redo months of frontier progress.
“Anthropic has an internal benchmark called CoBench, which contains around 500 real problems that its researchers and engineers solved this year.”
Choosing which objective to verify, which hill for GPT-7 to climb, is itself a key part of AI research called research taste, and before 2018 almost nobody thought minimizing next-token prediction loss was the most tractable path.
“figuring out what objective to verify in the first place—choosing which hill to have GPT-7 climb—is itself an important part of AI research. This is what one might call “research taste.””
AI R&D has three broad buckets, pretraining, post-training, and inference efficiency, each with strong verifiable proxies, and since together they comprise all algorithmic progress, AI progress is generally easy to train against.
“each has strong verifiable proxies that are easy to train against.”
Patel assigns roughly a 25% chance that the milestone of automated top AI researchers does not happen until after 2032, because some important part of AI research remains hard to automate.
“I’d guess there’s a 25% chance that it doesn’t happen until after 2032.”
Patel's median estimate is that automated researchers will match the very best human AI researchers in 2028, assuming labs do not intentionally slow down.
“If I had to pick my median year for when automated researchers will be as good as the very best human AI researchers (and assuming the labs don’t intentionally slow down), I’d guess 2028.”
Plugging METR's finding that Anthropic's rate of AI progress doubled in a year into a takeoff model straightforwardly implies that the milestone could be reached in 2027.
“2027 is also the straightforward implication of plugging METR’s finding—that Anthropic’s rate of AI progress has doubled in a year—into a takeoff model.”
Patel now believes AIs as good as the best human AI researchers are only a couple of years away, and possibly only a couple of months away, after updating sharply over the last three months.
“I’ve now come to the view that we’re only a couple of years away, and possibly only a couple of months away, from AIs that are as good as the best humans at AI research.”
Adding parallel researchers yields steeply diminishing returns, so 100,000 collaborators will not be 100,000 times faster, probably not even 100 times faster, but orders of magnitude more automated labor should still make progress at least a couple of times faster.
“But it’s implausible that you wouldn’t be going at least a couple of times faster from having orders of magnitude more automated labor.”
Data will keep growing during RSI but AIs will compile it, since producing better pretraining and post-training data is itself an ML research problem that AIs are getting increasingly good at.
“My answer is that data will continue growing during RSI, but the AIs will be the ones compiling it.”
The bottlenecks to RSI are serious and tamp down the probability of the most aggressive takeoff, which is why Patel expects a GPT-3.5-to-GPT-6-sized jump in 2028 rather than an overnight singularity, but they do not fully deflate the craziness of the future because the neck of the bottle is wide.
“But I think these bottlenecks don’t fully deflate the craziness of this future (and the neck of the bottle is actually pretty wide).”
An attention-based architecture does not outperform an LSTM unless it also has residual connections, layer norm, the Adam optimizer, intermixed MLP layers, positional encodings, and learning-rate warmup.
“an attention-based architecture does not perform better than an LSTM unless you also have residual connections (2015), layer norm (2016), the Adam optimizer (2014)”
Even the spark scenario, in which about four years of progress happens in the first year after automating AI research, would be comparable to the jump from GPT-3.5 to a GPT-6-level model in 12 months.
“That's a jump as big as GPT-3.5 to GPT-6 Astra in just 12 months.”
Pretraining data improvements are largely normal ML research, basically figuring out which filtering or synthetic data techniques lower loss on a held-out corpus, and open corpora grew from roughly 9B tokens in OpenWebText to far larger, classifier-filtered scrapes like UltraFineWeb.
“basically figuring out which techniques for filtering existing data or generating new synthetic data empirically lead to lower loss on a held-out corpus.”
Continual learning turned out to be harder than solving Navier-Stokes, as shown by labs not yet cracking it, which means expectations about which problem is harder were wrong.
“It turns out that continual learning was harder, given the fact that the labs haven't cracked it yet.”
Environment construction is becoming automated, with pipelines using difficulty and correctness as reward signals to iteratively train models to construct better tasks.
“we invest in large-scale, automated pipelines for data synthesis and environment construction”
The final model in the original 2017 transformer paper took 3.5 days on 8 P100 GPUs, which costs about 50 dollars to rent today, roughly one ten-thousandth of GPT-3's training compute.
“the final model trained in the original transformer paper back in 2017 took 3.5 days on 8 P100s—which costs about 50 bucks to rent today”
Inference efficiency is especially verifiable, since GPU utilization, KV cache size per token, total FLOPs in a forward pass, and latency can all be measured easily.
“You can easily measure GPU utilization, KV cache size per token, total FLOPs in a forward pass, latency, etc.”
Some parts of AI research are less verifiable than math, which is why OpenAI has not fired all its researchers, but AI research is verifiable enough that AIs will soon match the best humans at it.
“I want to clarify that some parts of AI research are, in fact, less verifiable than math.”
Current models are solving Millennium Prize problems while still being savant interns without long-term memory, and if they could actually run, given more capability, they would be faster, like a cartwheeling racer still keeping pace.
“These models are solving Millennium Prize problems while still being derpy savant interns who have no long-term memory.”
Debugging is trainable: you can inject subtle bugs into a training run and reward the model for finding them.
“Debugging is trainable too: You can inject subtle bugs into a training run and reward the model for finding them.”
Ajeya Cotra argued that if AI is nearly doubling the rate of AI capabilities progress and that acceleration is smooth, naive modeling implies reaching extreme superintelligence within a year, though Patel thinks this modeling is uncertain and his median is later.
“this naively implies that we would reach extreme superintelligence within a year.”
In Patel's own evaluation of models playing the mafia-style social deception game Blood on the Clocktower, more recently released models far outperformed earlier ones on strategic reasoning, long-term coherence, and multi-agent coordination.
“Models released more recently absolutely demolished the models released earlier this year”
Pressed for a probability, Patel gives a 50% chance to the spark scenario and a 50% chance to the software-only singularity.
“If pressed, I’d give a 50% chance to the “spark” scenario and a 50% chance to the “software-only singularity.””