What this covers

The episode is a single-narrator argument from Dwarkesh Patel that starts with mathematics. AI went from failing four-digit addition in 2023 to an IMO gold medal in 2025, then to open Erdős problems and, by his account, a Millennium Prize problem solved in 88 hours. He treats this as a template for what happens wherever answers can be checked, and asks how much of AI research is that kind of verifiable domain. His evidence includes METR's finding, after three months inside Anthropic, that the lab's rate of progress on well-scoped tasks has doubled. He also points to held-out loss as a direct measure of pretraining progress, and to Magic's claim of a base model for roughly $500,000.

The middle of the piece takes up the objections. Patel discusses reinforcement learning in datacenters versus real-world work like running businesses or winning elections, and the lack of continual learning, where every session is a first day on the job. He bets that longer in-context sessions may be enough. He also argues about research taste: the transformer was a combination of known ideas, evolution produced a general learner from a crude verifier, and Toby Ord doubts AI can yet pose powerful new questions.

The closing stretch is quantitative. Ideas are getting harder to find, with R&D compute quadrupling and staff doubling each year just to hold pace. Automated researchers would run about 50 times more work per calendar week than a human, though compute limits bite hard. Global compute grows about 3x a year, so the endpoint arrives over years rather than months. DeepSeek is cited as finding that data and environment quality matter more than RL algorithm changes.

Sharpest takeaway

Patel argues that AI research is highly verifiable, so AI is already accelerating its own progress, and automated AI researchers could match the best human researchers by around 2028 (possibly 2027). That would trigger either a software-only singularity or a 'spark' that compresses several years of progress into one year.

  • METR and Anthropic/OpenAI usage data show AI already speeds up frontier AI R&D.
  • Math progress shows AIs excel at verifiable, well-scoped search-and-match discovery, and AI research has similar verifiable proxies.
  • Bottlenecks like continual learning, data, and experiments slow but do not cancel the takeoff.

The claims · ranked77 claims · weighted by value

This asset isn't compiled yet

You're seeing its claims, ranked. Compile it to build the argument threads, weight them, and check each claim against your library — the full view.

0.78

Automated researchers could develop a much better science of learning from small experiments, extending scaling laws and discovering phenomena like muP, which predicts optimal hyperparameters for larger runs from models 100x smaller, and GPT-4's loss was predicted from models trained with 10,000x less compute.

factualhigh valueestablishednovelty 3/4durability 3/4· Dwarkesh Patel

“the GPT-4 paper says that they were able to predict that model’s final loss using models trained with 10,000x less compute.”

0.75

Even without AI help, global AI compute has been growing roughly 3x per year, and spark-era AIs would help build more chips, datacenters, and robots, so reaching superintelligence takes years rather than months, but the endpoint is similar.

factualhigh valueestablishednovelty 2/4durability 3/4· Dwarkesh Patel

“Even without help from AIs, global AI compute has been growing roughly 3x a year.”

0.75

AI companies have had to quadruple their R&D compute and double their workforce every single year just to maintain the same rate of progress, showing that ideas are getting harder to find.

factualhigh valueestablishednovelty 2/4durability 3/4· Dwarkesh Patel

“AI companies have to quadruple their R&D compute and double their workforce every single year just to maintain the same rate of progress.”

0.74

Evolution optimized humans using a very crude verifier, whether genes were passed on, with ancestral tasks like hunting and tribal standing, yet it produced a general-purpose learner; deliberate AI training with better-matched tasks should therefore transfer even further.

causalhigh valuecontestednovelty 3/4durability 4/4· Dwarkesh Patel

“Our general-purpose mental architecture was developed by evolution, which is an optimization process with a really crude verifier: Did you pass your genes on or not?”

0.72

DeepSeek reports that changes to the basic RL algorithm have not mattered nearly as much as systematic improvements in the scale, diversity, and verifiability of synthesized data and environments, which account for essentially all observed gains.

factualhigh valueestablishednovelty 3/4durability 2/4· Dwarkesh Patel

“systematic improvements in the scale, diversity, and verifiability of synthesized data and environments account for essentially all of the observed gains.”

0.70

Proxies made before a paradigm leap still capture large step changes because AI learning algorithms are general: the transformer was visible on the translation benchmark it was invented to beat, and paradigm-agnostic outer-loop proxies like unseen-game mastery or pre-1905 physics discovery remain available.

causalhigh valuecontestednovelty 3/4durability 3/4· Dwarkesh Patel

“The transformer was visible on (in fact was invented to beat) the existing translation benchmark”

0.69

METR, after three months embedded inside Anthropic, concluded that Anthropic's overall rate of progress on well-scoped tasks is 2x what it was a year ago, and that a majority of this acceleration is likely attributable to Anthropic's extensive use of AI to do R&D.

factualhigh valuecontestednovelty 3/4durability 2/4· Dwarkesh Patel

“Anthropic’s overall rate of progress on well-scoped tasks is 2x what it was a year ago, and that a “majority of this acceleration today is likely attributable to Anthropic’s extensive use of AI to do R&D.””

0.68

Today's AIs already work at about 10 times human speed, and because automated researchers would work around the clock, each one would cram roughly 50 times more work into every calendar week than a human researcher, though compute bottlenecks would bring steep diminishing returns.

factualhigh valueestablishednovelty 2/4durability 3/4· Dwarkesh Patel

“each individual automated researcher will be cramming in roughly 50x more work every calendar week than a human researcher can do.”

0.68

The transformer was a natural combination of ideas, such as attention and residual networks, that had independently been effective, so none of its core pieces were surprising to researchers in the field.

factualhigh valueestablishednovelty 2/4durability 3/4· Dwarkesh Patel

“the transformer architecture was a natural combination of many ideas (attention, residual networks) which had independently been effective”

0.68

Current AI is trained with RL on tasks that are easy to grade within a datacenter, but most of the world, such as building businesses, winning court cases, trading, and winning elections, requires real-world interaction that cannot be recreated inside a datacenter.

factualhigh valuecontestednovelty 2/4durability 3/4· Dwarkesh Patel

“The way we’re currently making AIs smarter is by RLing them on tasks that are easy to automatically grade from within a datacenter”

0.68

Sample efficiency and online learning are two sides of the same problem of learning from real-world data quickly, a deep open problem in machine learning that is AI's equivalent of a Millennium Prize problem, but every kind of sample efficiency is measurable and therefore verifiable.

definitionhigh valueestablishednovelty 2/4durability 3/4· Dwarkesh Patel

“Sample efficiency and online learning are really two sides of the same problem: how to learn from real-world data quickly and efficiently.”

0.68

Pretraining progress is extremely easy to verify because loss on a held-out test set is a direct measure, so GPT-7 could be RL-trained to minimize compute needed to reach a fixed loss, and swarms could search for the lowest perplexity.

factualhigh valueestablishednovelty 2/4durability 3/4· Dwarkesh Patel

“Loss on a held-out test set, which is the basis of pretraining progress, is extremely easy to verify.”

0.66

Because models generalize far beyond what they were trained on, models trained on less ambitious verifiable math problems can solve problems no human had cracked, so the same generalization should make models excellent at unverifiable parts of AI research like research taste.

causalhigh valuecontestednovelty 3/4durability 3/4· Dwarkesh Patel

“models generalize far beyond what they were directly trained on”

0.66

Before the 2017 transformer paper, attention, residual networks and other ideas were already known, and the transformer was only obviously better once half a dozen additional tweaks were combined, which required research taste and many small experiments.

causalhigh valuecontestednovelty 3/4durability 3/4· Dwarkesh Patel

“One reason was that this new architecture was only obviously better than the state of the art once you connected half a dozen additional tweaks and innovations.”

0.66

AI math capability went from failing at four-digit addition in 2023, to barely solving high school competition problems in 2024, to winning an IMO gold medal in 2025, to solving open Erdős problems, and about a month before recording solving a Millennium Prize problem in 88 hours.

factualhigh valueestablishednovelty 2/4durability 2/4· Dwarkesh Patel

““Okay, it fuckingsolved a Millennium Prize problemin 88 hours.””

0.65

Pretraining improvements yield very general gains in intelligence because of a deep link between compression and intelligence, so a pretraining-automation loop makes the model broadly better.

causalhigh valuecontestednovelty 2/4durability 4/4· Dwarkesh Patel

“There is a deep explanation for why this is true, which has to do with the link between compression and intelligence”

0.65

Scaling in-context learning so sessions last many months may be enough for a single AI instance to learn on the job effectively, without an architectural breakthrough that updates weights at the end of each session.

forecasthigh valuecontestednovelty 3/4durability 2/4· Dwarkesh Patel

“scaling up in-context learning, so that you have sessions that are effectively many months long, will probably be enough to let a single instance learn on the job effectively.”

0.65

Patel does not think every real-world answer can be baked into model weights; rather, scaling up RL training will give AIs the abilities to ramp quickly in new contexts, even in hard-to-verify domains like running businesses, court cases, trading, or elections.

forecasthigh valuecontestednovelty 3/4durability 2/4· Dwarkesh Patel

“my claim is that scaling up RL training will give AIs the abilities necessary to ramp quickly in new contexts, even in hard-to-verify domains”

0.64

In every domain where answers can be checked and AIs can practice thousands of rounds, such as competitive programming and well-scoped open math problems, models have gone from worse than most humans to better than the best humans within a few years.

factualhigh valuecontestednovelty 2/4durability 3/4· Dwarkesh Patel

“In every domain where we can check answers and give AIs thousands of rounds of practice ... they've gone from worse than most humans to better than the best humans within a few years.”

0.63

In verifiable domains like mathematics, AIs are doing search-and-match discovery so effectively that hundreds of important results, which top mathematicians had spent decades trying to resolve, are being produced as side quests to hype future model releases.

factualhigh valuecontestednovelty 2/4durability 2/4· Dwarkesh Patel

“hundreds of important results (which the top mathematicians and computer scientists in the world had spent decades trying to resolve) are being churned out as side quests”

0.63

Current AIs lack the ability to improve from on-the-job experience: every new session is effectively a first day on the job, and each session retains only a couple of hours of information, so they do not accumulate weeks or months of feedback.

factualhigh valueestablishednovelty 2/4durability 2/4· Dwarkesh Patel

“Every new session is their first day on the job. And in each session, they can only retain a couple of hours’ worth of information”

0.63

The AI lab Magic announced it pretrained a model matching leading open-source base models such as DeepSeek V4 Pro for just half the compute used to train the original GPT-3, roughly 500,000 dollars today, and Patel thinks orders-of-magnitude further improvements are plausible.

factualhigh valueestablishednovelty 2/4durability 2/4· Dwarkesh Patel

“they pretrained a model that matches the leading open-source base models today, like DeepSeek V4 Pro, for just half of the compute used to train the original GPT-3”

0.63

OpenAI said GPT-5.2, which is dumber and less efficient than today's models, completed GDPval tasks 11 times faster than human experts, and a frontier model outputs about 60 tokens per second, roughly 13 times faster than a human talks.

factualhigh valueestablishednovelty 2/4durability 2/4· Dwarkesh Patel

“OpenAI said that GPT-5.2, which is dumber and less efficient than today’s models, completed the tasks on its GDPval benchmark 11 times faster than human experts.”

0.62

Math progress has skewed disproportionately toward verification-laden parts like finding counterexamples rather than creating new theories, and Toby Ord argues there is little evidence AI can ask powerful questions or find profound new fields of mathematical inquiry.

factualhigh valuecontestednovelty 3/4durability 2/4· Dwarkesh Patel

“It’s true that progress has skewed disproportionately towards the verification-laden parts of math, like finding counterexamples (rather than coming up with new theories, say).”

0.61

As of August, Claude 'leads' more than 26% of Anthropic's AI R&D work, up from under 1% in February; 'leads' means completing most of a task end-to-end from a high-level prompt with a human supervising, while fully autonomous work remains at 0%.

factualhigh valueestablishednovelty 2/4durability 1/4· Dwarkesh Patel

“as of August, Claude “leads” more than 26% of Anthropic’s AI R&D work—up from under 1% as recently as February.”

0.61

OpenAI reported that the heaviest 10% of its researchers' AI users went from using around $1,000 a day of internal coding agents at the start of summer to over $7,000 a day by the end of August.

factualhigh valueestablishednovelty 2/4durability 1/4· Dwarkesh Patel

“the heaviest 10% of AI users among OpenAI’s researchers went from using around $1,000 a day of internal coding agents at the beginning of the summer to over $7,000 a day by the end of August.”

0.56

All the progress made in deep learning over the last decade and a half has not required much new theory, meaning empirical recipe-finding has driven the field.

factualhigh valuecontestednovelty 2/4durability 2/4· Dwarkesh Patel

“all the progress we've made in deep learning over the last decade and a half has not required much new theory anyway”

0.56

Even at one ten-thousandth of the RL training scale expected during the intelligence explosion, current models already generalize: Claude Code or Codex can orient to an unfamiliar project, figure out how it fits together, and tackle an ambitious assignment.

factualhigh valuecontestednovelty 2/4durability 2/4· Dwarkesh Patel

“even at one ten-thousandth of the scale of RL training that will be happening during the intelligence explosion, we’re already seeing lots of generalization.”

0.56

Research taste is itself verifiable: an RL environment can ask GPT-7 to predict whether an algorithmic improvement made at 100M parameters will hold at 10B, a taste-bench that is costly but feasible on frontier compute.

factualhigh valuespeaker onlynovelty 4/4durability 3/4· Dwarkesh Patel

“you can make an RL environment where GPT-7 has to predict whether an algorithmic improvement made at the 100M-parameter scale will hold at the 10B scale.”

0.55

A similar jump starting in 2027 or 2028 would produce a mind vastly superhuman in slightly verifiable domains, better than Nvidia at designing next-generation AI accelerators, better than Citadel or Jane Street at trading if given equal market access, and better than all humanity at finding a room-temperature superconductor given lab access.

forecasthigh valuespeaker onlynovelty 4/4durability 2/4· Dwarkesh Patel

“you’d probably get a mind that would be vastly superhuman in domains that are even slightly verifiable.”

0.54

Well-scoped problem solving matters even more for machine learning research than for mathematics, because mathematics values new theory for its own sake while AI labs care mainly about whether an algorithmic recipe empirically improves training efficiency or model smartness.

causalhigh valuespeaker onlynovelty 3/4durability 3/4· Dwarkesh Patel

“But in AI research, the priorities are exactly reversed. AI labs mainly care about how any new algorithmic recipe empirically makes training more efficient or elicits smarter models.”

0.54

Automated researchers can trade labor for compute: where human researchers waste compute to save scarce researcher time, agents can run many more small ablations, read every rollout, and rebuild broken environments, so experiment compute is used far more effectively than today.

causalhigh valuespeaker onlynovelty 3/4durability 3/4· Dwarkesh Patel

“Immediately, they can trade labor for compute. To save valuable researcher time, experiments today are, and should be, very wasteful with compute.”

0.54

AI research is closer to formalizing a theorem than writing a novel, because it contains many independent modular subtasks with verifiable proxies, so a swarm of agents can try a thousand approaches per part of the pipeline and speed progress a lot.

causalhigh valuespeaker onlynovelty 3/4durability 3/4· Dwarkesh Patel

“AI research is closer to formalizing a theorem than to writing a novel, because there is a lot of independent modular work to do, and there are verifiable proxies for progress.”

0.54

Automating AI research will compress so much progress into a single year that, soon thereafter (by 2030, plausibly as early as next year), AIs will be at least as good as the best human experts in every field.

forecasthigh valuespeaker onlynovelty 4/4durability 1/4· Dwarkesh Patel

“automating AI research will compress so much progress into a single year that, soon thereafter (so by 2030, but plausibly as early as next year), we’ll have AIs that are at least as good as the best human experts in every field.”

0.54

A SlothAI with the same compute but researchers working one hour a week would make algorithmic progress at least five times slower, and since automated companies are to OpenAI as OpenAI is to SlothAI, human thinking speed is a biological accident that sets no special limit on useful thinking speed.

causalhigh valuespeaker onlynovelty 3/4durability 3/4· Dwarkesh Patel

“It seems to me that, intuitively, SlothAI would end up making algorithmic progress at least five times slower.”

0.54

If ideas get harder to find faster than recursive self-improvement makes AIs better at research, the result is a 'spark' of roughly three to six years of AI progress in a single year, followed by a slowdown until more compute is built.

forecasthigh valuespeaker onlynovelty 3/4durability 3/4· Dwarkesh Patel

“we’ll end up with what I’d call a “spark.” We’d get something like three to six years of AI progress in a single year.”

0.53

During the intelligence explosion, automated engineers will build tens of thousands of times more RL training environments than exist today, including long-running simulated economies where AIs cofound companies through centuries of simulated corporate competition.

forecasthigh valuespeaker onlynovelty 3/4durability 2/4· Dwarkesh Patel

“During the intelligence explosion, automated engineers will build tens of thousands of times more RL training environments than we have currently.”

0.53

An AI shadowing a CEO could read every email, Slack message and document on its first morning, and parallel copies could interview every employee and major client, doing a human's months of work by the end of day one, so AIs could reach top human performance in most domains within months of deployment, though feedback on quarterly results requires serial time.

forecasthigh valuespeaker onlynovelty 3/4durability 2/4· Dwarkesh Patel

“By the end of the first day, parallel copies of it could have interviewed every employee and major client, and cumulatively done the amount of work it would have taken a single human months to do”

0.52

AIs can train on prediction tasks built from history, such as deriving general relativity from a corpus ending before 1700 or predicting the Cuban Missile Crisis from newspapers through 1962, developing scientific and political judgment beyond the shrewdest human forecasters.

forecasthigh valuespeaker onlynovelty 4/4durability 2/4· Dwarkesh Patel

“Can an AI figure out general relativity given a training corpus that includes no information beyond 1700”

0.52

For pretraining over the last six years, more than three times as much algorithmic progress has come from data improvements as from model improvements, according to research Patel did with Jerry Han.

factualhigh valuespeaker onlynovelty 4/4durability 2/4· Dwarkesh Patel

“we found that for pretraining, over the last six years, more than three times as much algorithmic progress has come from data improvements as from model improvements.”

0.52

Automating pretraining would radically speed AI progress for three reasons: pretraining gives general intelligence gains, smaller models trained faster can match larger ones and support more researchers, and faster small-model training tightens the RSI loop because giant runs become obsolete before finishing.

causalhigh valuespeaker onlynovelty 2/4durability 3/4· Dwarkesh Patel

“By automating pretraining alone, you’d radically speed up AI progress.”

0.52

Getting good at a complex task requires two things: domain-specific knowledge and a more general set of skills like seeking information, making plans, persisting through mistakes, tracking what you've learned, and collaborating with others.

definitionhigh valuespeaker onlynovelty 3/4durability 3/4· Dwarkesh Patel

“Getting really good at any complex task takes two things. One is domain-specific knowledge. The other is a more general set of skills”

0.52

Today's experiment practices reflect that researcher time is extremely scarce relative to compute, since top researchers earn hundreds of millions a year, so current compute use tells us little about how efficiently labs could use compute if compute were the key constraint.

causalhigh valuespeaker onlynovelty 3/4durability 3/4· Dwarkesh Patel

“researcher time is extremely scarce relative to compute—top AI researchers individually get paid hundreds of millions of dollars a year.”

0.52

As pretraining improves, smaller models can match larger ones while training faster and cheaper, which speeds experiments and lets labs train models at a scale where thousands of experiments run daily, so small-scale results need not be bet on bigger models.

causalhigh valuespeaker onlynovelty 3/4durability 3/4· Dwarkesh Patel

“because smaller models train faster, this tightens the RSI loop itself.”

0.52

If the rate at which AIs get smarter outpaces the rate at which ideas get harder to find, a software-only singularity follows, where without more compute the AI swarm produces a superintelligence scraping against the physical limits of current chips within months.

forecasthigh valuespeaker onlynovelty 3/4durability 3/4· Dwarkesh Patel

“we’d get something called the “software-only singularity.” In this scenario, even without increasing compute, the AI swarm rapidly makes smarter and smarter versions of itself”

0.52

AI research resembles the mathematical progress AIs are good at: unlocking large amounts of low-hanging fruit through persistence and a superhuman ability to connect ideas from many adjacent domains.

causalhigh valuespeaker onlynovelty 3/4durability 3/4· Dwarkesh Patel

“AI research is actually surprisingly similar to the type of mathematical progress that current AIs are disproportionately good at: unlocking a ton of low-hanging fruit by applying a combination of incredible persistence and a superhuman capacity to connect ideas from lots of adjacent domains.”

0.52

Napoleon at 27 and Rockefeller at 25 ramped up in domains they were new to, needing only a couple years of experience rather than decades, so AIs should be able to replicate or exceed that learning speed.

causalhigh valuespeaker onlynovelty 3/4durability 3/4· Dwarkesh Patel

“Rockefeller and Napoleon needed experience, yes, but only a couple of years’ worth, not decades.”

0.51

Choosing which objective to optimize is part of research taste, but paradigm-agnostic outer-loop proxies such as Millennium Prize solves, time to rediscover general relativity from pre-1905 data, and speed of mastering unseen games can verify progress across paradigms.

factualhigh valuespeaker onlynovelty 3/4durability 2/4· Dwarkesh Patel

“It's not that difficult to come up with paradigm-agnostic outer loop proxies”

0.51

Frontier labs quadruple compute yearly, so by the end of 2027 OpenAI could give thousands of agents enough compute each to run a GPT-3-sized training run daily, and the trade of labor for compute means experiment bottlenecks are weaker than they look.

forecasthigh valuespeaker onlynovelty 3/4durability 2/4· Dwarkesh Patel

“If this trend continues, then by the end of 2027, OpenAI could give thousands of agents each enough compute to run a GPT-3-sized training run every single day.”

0.51

Because online learning has clean verifiable proxies, models a year or two smarter than current ones might crack continual learning even without persistent memory, and it was not obvious in advance that continual learning would be harder than Navier-Stokes.

forecasthigh valuespeaker onlynovelty 3/4durability 2/4· Dwarkesh Patel

“But are you so confident that it’s that much harder that non-online-learning models a year or two smarter than the ones currently solving Millennium Prize problems won’t be able to crack it”

0.51

Around 80% of the algorithmic progress at a given frontier lab is driven by its top 50 researchers, so automated researchers matching the very best humans would yield a large additional speedup.

factualhigh valuespeaker onlynovelty 3/4durability 2/4· Dwarkesh Patel

“My sense is that around 80% of the algorithmic progress at a given frontier lab is driven by its top 50 researchers.”

0.51

Automated researchers will radically improve learning algorithms through online learning and sample efficiency breakthroughs, and future AIs could be far beyond humans in learning efficiency, since humans learn from data far more efficiently than LLMs do.

forecasthigh valuespeaker onlynovelty 3/4durability 2/4· Dwarkesh Patel

“In the same way that humans are much more efficient and effective at learning from data than current LLMs are, future AIs could be far beyond humans in learning efficiency.”

0.49

Because matching dozens of idea combinations is what AIs are disproportionately skilled at, a model like Claude 7 could invent something as large as the transformer if Chris Olah asked it to hard enough to improve attention.

forecasthigh valuespeaker onlynovelty 4/4durability 1/4· Dwarkesh Patel

“is the kind of thing that Claude 7 could invent if Chris Olah asked it to try hard to get this attention thing to work better.”

0.48

Current AI systems have solved a Millennium Prize problem without the ability to ask powerful questions or find profound new fields of mathematical inquiry, so well-scoped problem solving can be a major capability independent of theory-generation.

causalhigh valuespeaker onlynovelty 3/4durability 2/4· Dwarkesh Patel

“Current AI systems have solved a Millennium Prize problem, all without the ability to ask powerful questions”

0.48

Post-training has verifiable proxies: compute or data needed to RL on new environments, how quickly a model learns unseen games or tasks, and benchmarks like Anthropic's CoBench of around 500 real R&D problems, which can become RL environments that let models redo months of frontier progress.

factualhigh valuespeaker onlynovelty 3/4durability 2/4· Dwarkesh Patel

“Anthropic has an internal benchmark called CoBench, which contains around 500 real problems that its researchers and engineers solved this year.”

0.47

Choosing which objective to verify, which hill for GPT-7 to climb, is itself a key part of AI research called research taste, and before 2018 almost nobody thought minimizing next-token prediction loss was the most tractable path.

definitionhigh valuespeaker onlynovelty 2/4durability 3/4· Dwarkesh Patel

“figuring out what objective to verify in the first place—choosing which hill to have GPT-7 climb—is itself an important part of AI research. This is what one might call “research taste.””

0.47

AI R&D has three broad buckets, pretraining, post-training, and inference efficiency, each with strong verifiable proxies, and since together they comprise all algorithmic progress, AI progress is generally easy to train against.

factualhigh valuespeaker onlynovelty 2/4durability 3/4· Dwarkesh Patel

“each has strong verifiable proxies that are easy to train against.”

0.46

Patel assigns roughly a 25% chance that the milestone of automated top AI researchers does not happen until after 2032, because some important part of AI research remains hard to automate.

forecasthigh valuespeaker onlynovelty 2/4durability 2/4· Dwarkesh Patel

“I’d guess there’s a 25% chance that it doesn’t happen until after 2032.”

0.45

Patel's median estimate is that automated researchers will match the very best human AI researchers in 2028, assuming labs do not intentionally slow down.

forecasthigh valuespeaker onlynovelty 3/4durability 1/4· Dwarkesh Patel

“If I had to pick my median year for when automated researchers will be as good as the very best human AI researchers (and assuming the labs don’t intentionally slow down), I’d guess 2028.”

0.45

Plugging METR's finding that Anthropic's rate of AI progress doubled in a year into a takeoff model straightforwardly implies that the milestone could be reached in 2027.

forecasthigh valuespeaker onlynovelty 3/4durability 1/4· Dwarkesh Patel

“2027 is also the straightforward implication of plugging METR’s finding—that Anthropic’s rate of AI progress has doubled in a year—into a takeoff model.”

0.45

Patel now believes AIs as good as the best human AI researchers are only a couple of years away, and possibly only a couple of months away, after updating sharply over the last three months.

forecasthigh valuespeaker onlynovelty 3/4durability 1/4· Dwarkesh Patel

“I’ve now come to the view that we’re only a couple of years away, and possibly only a couple of months away, from AIs that are as good as the best humans at AI research.”

0.44

Adding parallel researchers yields steeply diminishing returns, so 100,000 collaborators will not be 100,000 times faster, probably not even 100 times faster, but orders of magnitude more automated labor should still make progress at least a couple of times faster.

forecastestablishednovelty 2/4durability 2/4· Dwarkesh Patel

“But it’s implausible that you wouldn’t be going at least a couple of times faster from having orders of magnitude more automated labor.”

0.44

Data will keep growing during RSI but AIs will compile it, since producing better pretraining and post-training data is itself an ML research problem that AIs are getting increasingly good at.

forecasthigh valuespeaker onlynovelty 2/4durability 2/4· Dwarkesh Patel

“My answer is that data will continue growing during RSI, but the AIs will be the ones compiling it.”

0.44

The bottlenecks to RSI are serious and tamp down the probability of the most aggressive takeoff, which is why Patel expects a GPT-3.5-to-GPT-6-sized jump in 2028 rather than an overnight singularity, but they do not fully deflate the craziness of the future because the neck of the bottle is wide.

normativehigh valuespeaker onlynovelty 2/4durability 2/4· Dwarkesh Patel

“But I think these bottlenecks don’t fully deflate the craziness of this future (and the neck of the bottle is actually pretty wide).”

0.42

An attention-based architecture does not outperform an LSTM unless it also has residual connections, layer norm, the Adam optimizer, intermixed MLP layers, positional encodings, and learning-rate warmup.

factualestablishednovelty 2/4durability 3/4· Dwarkesh Patel

“an attention-based architecture does not perform better than an LSTM unless you also have residual connections (2015), layer norm (2016), the Adam optimizer (2014)”

0.40

Even the spark scenario, in which about four years of progress happens in the first year after automating AI research, would be comparable to the jump from GPT-3.5 to a GPT-6-level model in 12 months.

forecasthigh valuespeaker onlynovelty 2/4durability 1/4· Dwarkesh Patel

“That's a jump as big as GPT-3.5 to GPT-6 Astra in just 12 months.”

0.36

Pretraining data improvements are largely normal ML research, basically figuring out which filtering or synthetic data techniques lower loss on a held-out corpus, and open corpora grew from roughly 9B tokens in OpenWebText to far larger, classifier-filtered scrapes like UltraFineWeb.

factualestablishednovelty 2/4durability 2/4· Dwarkesh Patel

“basically figuring out which techniques for filtering existing data or generating new synthetic data empirically lead to lower loss on a held-out corpus.”

0.36

Continual learning turned out to be harder than solving Navier-Stokes, as shown by labs not yet cracking it, which means expectations about which problem is harder were wrong.

factualestablishednovelty 2/4durability 2/4· Dwarkesh Patel

“It turns out that continual learning was harder, given the fact that the labs haven't cracked it yet.”

0.36

Environment construction is becoming automated, with pipelines using difficulty and correctness as reward signals to iteratively train models to construct better tasks.

factualestablishednovelty 2/4durability 2/4· Dwarkesh Patel

“we invest in large-scale, automated pipelines for data synthesis and environment construction”

0.36

The final model in the original 2017 transformer paper took 3.5 days on 8 P100 GPUs, which costs about 50 dollars to rent today, roughly one ten-thousandth of GPT-3's training compute.

factualestablishednovelty 2/4durability 2/4· Dwarkesh Patel

“the final model trained in the original transformer paper back in 2017 took 3.5 days on 8 P100s—which costs about 50 bucks to rent today”

0.35

Inference efficiency is especially verifiable, since GPU utilization, KV cache size per token, total FLOPs in a forward pass, and latency can all be measured easily.

factualestablishednovelty 1/4durability 3/4· Dwarkesh Patel

“You can easily measure GPU utilization, KV cache size per token, total FLOPs in a forward pass, latency, etc.”

0.26

Some parts of AI research are less verifiable than math, which is why OpenAI has not fired all its researchers, but AI research is verifiable enough that AIs will soon match the best humans at it.

normativespeaker onlynovelty 2/4durability 2/4· Dwarkesh Patel

“I want to clarify that some parts of AI research are, in fact, less verifiable than math.”

0.26

Current models are solving Millennium Prize problems while still being savant interns without long-term memory, and if they could actually run, given more capability, they would be faster, like a cartwheeling racer still keeping pace.

factualspeaker onlynovelty 2/4durability 2/4· Dwarkesh Patel

“These models are solving Millennium Prize problems while still being derpy savant interns who have no long-term memory.”

0.26

Debugging is trainable: you can inject subtle bugs into a training run and reward the model for finding them.

factualspeaker onlynovelty 2/4durability 2/4· Dwarkesh Patel

“Debugging is trainable too: You can inject subtle bugs into a training run and reward the model for finding them.”

0.22

Ajeya Cotra argued that if AI is nearly doubling the rate of AI capabilities progress and that acceleration is smooth, naive modeling implies reaching extreme superintelligence within a year, though Patel thinks this modeling is uncertain and his median is later.

forecastspeaker onlynovelty 2/4durability 1/4· Dwarkesh Patel

“this naively implies that we would reach extreme superintelligence within a year.”

0.18

In Patel's own evaluation of models playing the mafia-style social deception game Blood on the Clocktower, more recently released models far outperformed earlier ones on strategic reasoning, long-term coherence, and multi-agent coordination.

factualspeaker onlynovelty 1/4durability 1/4· Dwarkesh Patel

“Models released more recently absolutely demolished the models released earlier this year”

0.18

Pressed for a probability, Patel gives a 50% chance to the spark scenario and a 50% chance to the software-only singularity.

forecastspeaker onlynovelty 1/4durability 1/4· Dwarkesh Patel

“If pressed, I’d give a 50% chance to the “spark” scenario and a 50% chance to the “software-only singularity.””