An audio version of my blog post, Thoughts on AI progress (Dec 2025)
What this covers
Dwarkesh Patel makes the case that the field has mistaken benchmark progress for economic progress and is chasing the wrong scaling approach. The core argument rests on a distinction between how models improve on test suites versus how useful they become in real work. RLVR scaling—the practice of training models through reinforcement learning on verifiable outcomes—lacks the clean, law-like trend that pretraining scaling exhibited, yet advocates have borrowed pretraining's credibility to project optimistic timelines. More fundamentally, Patel contends that baking skills into models through custom training pipelines is a confession that these systems cannot learn the way humans do: on the job, from feedback, across novel contexts. If true continual learning emerges, such pre-baking becomes pointless; if it doesn't, then AGI is not near.
The economic case sharpens the argument. Lab revenues run four orders of magnitude below knowledge-worker wages, yet if models were truly capable AGI-class systems, firms would spend trillions in tokens annually. This gap reflects not slow diffusion but genuine capability shortfall. Patel challenges the Flywheel Effect narrative as well, observing that supposed advantages like chat engagement and synthetic data have not produced runaway dominance—competition rotates monthly, suggesting no single lab has locked in advantage. Throughout, he distinguishes between what he calls real progress toward AGI and the incremental refinement of narrow tools. He grants that actual AGI—billions of human-like intelligences that can share and merge learning—would arrive within a decade or two and would be genuinely transformative, but argues the current path is not leading there. The disagreement is not whether AGI is possible, but whether it hinges on human-like learner capabilities that RLVR scaling is unlikely to produce.
Patel argues that current RL-on-LLMs scaling is fundamentally not the path to AGI because true AGI requires human-like continual learning and generalization, which means short timelines built on RLVR scaling are misguided, even though genuine AGI is likely within a decade or two.
- Baking skills into models via RL environments is unnecessary if models could learn on the job, and if they can't, AGI isn't imminent
- Lab revenues being ~4 orders of magnitude below knowledge-worker wages reveals the capability gap, not slow diffusion
- RLVR scaling lacks the clean predictable trend that pretraining scaling had, so bullish projections are borrowing pretraining's prestige
This asset isn't compiled yet
You're seeing its claims, ranked. Compile it to build the argument threads, weight them, and check each claim against your library — the full view.
Human workers are valuable precisely because you don't need to build schleppy custom training loops for every small part of their job; it's not net-productive to build a custom pipeline for each lab-specific micro-task, so what's actually needed is an AI that can learn from semantic feedback or self-directed experience and then generalize the way a human does.
“Human workers are valuable precisely because we don’t need to build schleppy training loops for every small part of their job. It’s not net-productive to build a custom training pipeline to identify what macrophages look like given the way this particular lab prepares slides, then another for the next lab-specific micro-task, and so on. What you actually need is an AI that can learn from semantic feedback or from self directed experience, and then generalize, the way a human does.”
Models keep getting more impressive at the rate the short-timelines people predict, but more useful at the rate the long-timelines people predict—capturing the divergence between benchmark progress and real economic utility.
“Models keep getting more impressive at the rate the short timelines people predict, but more useful at the rate the long timelines people predict.”
People are laundering the prestige of pretraining scaling—which showed a clean, near-physical-law improvement in loss across many orders of magnitude of compute—to justify bullish RLVR projections, even though RLVR has no well-fit publicly known trend and the scarce public datapoints yield bearish results.
“People are trying to launder the presitge of pretraining scaling, which was almost as predictable as a physical law of the universe, to justify bullish projections about RLVR, for which we have no well fit publicly known trend.”
Competition between model companies will stay fierce, informed by the observation that previous supposed flywheels (chat user engagement, synthetic data) have done little to diminish competition; the big three rotate around the podium roughly monthly, suggesting some force—talent poaching, rumor mills, or reverse engineering—has neutralized any runaway single-lab advantage.
“all these previous supposed flywheels (user engagement on chat, synthetic data, etc) have done very little to diminish the greater and greater competition between model companies. Every month (or less), the big three will rotate around the podium ... There is some force (potentially talent poaching, rumor mills, or reverse engineering) which has so far neutralized any runaway advantages a single lab might have had.”
Either models will soon learn on the job in a self-directed way, which makes the entire mid-training supply chain of pre-baking skills (like navigating browsers or using Excel) pointless, or they won't, which means AGI is not imminent; humans never go through a special phase rehearsing every piece of software they might use.
“Either these models will soon learn on the job in a self directed way - making all this pre-baking pointless - or they won’t - which means AGI is not imminent. Humans don’t have to go through a special training phase where they need to rehearse every single piece of software they might ever need to use.”
If models were truly like humans on a server they'd diffuse incredibly quickly—easier to onboard than human employees since they could read your entire Slack and Drive in minutes, distill all the skills of other AI employees, and avoid the lemons-market problem of human hiring where bad hires are costly and hard to detect.
“If these models were actually like humans on a server, they’d diffuse incredibly quickly. In fact, they’d be so much easier to integrate and onboard than a normal human employee (they could read your entire Slack and Drive in minutes and immediately distill all the skills your other AI employees have). Plus, hiring is very much like a lemons market, where it’s hard to tell who the good people are, and hiring someone bad is quite costly. This is a dynamic you wouldn’t have to worry about when you just wanna spin up another instance of a vetted AGI model.”
When frontier models improve at benchmarks, we should attribute it not just to increased scale and clever ML ideas but to billions of dollars spent paying PhDs, MDs, and other experts to write questions and example answers targeting those precise capabilities.
“When we see frontier models improving at various benchmarks we should think not just of increased scale and clever ML research ideas but billions of dollars spent paying PhDs, MDs, and other experts to write questions and provide example answers and reasoning targeting these precise capabilities.”
The diffusion-lag argument fails because highly-skilled, experienced, entrepreneurial immigrant humans integrate into the economy immediately, and since AGI will be able to do those same things, true AGI would diffuse rapidly rather than slowly.
“New technologies take a long time to integrate into the economy? Well ask yourself: how do highly-skilled, experienced, and entrepreneurial immigrant humans manage to integrate into the economy immediately? Once you’ve answered that question, note that AGI will be able to do those things too.”
Although AI bulls fairly criticize bears for moving the goal posts, some goal-post shifting is justified: we keep solving what we thought were sufficient bottlenecks to AGI (general understanding, few-shot learning, reasoning) yet still lack AGI, so the rational response is to conclude that prior definitions of AGI were too narrow and there is more to intelligence and labor than previously realized.
“We keep solving what we thought were the sufficient bottlenecks to AGI (general understanding, few shot learning, reasoning), and yet we still don’t have AGI (defined as, say, being able to completely automate 95% of knowledge work jobs). What is the rational response?”
Existing intelligence-explosion scenarios (software-only or software-plus-hardware singularities) neglect what will be the main driver of improvement atop AGI: continual learning, since humans become more capable mostly from experience in the relevant domain.
“All these scenarios neglect what I think will be the main driver of further improvements atop AGI: continual learning. Again, think about how humans become more capable at anything. It’s mostly from experience in the relevant domain.”
Connecting the dots between o-series benchmark charts, Toby Ord's analysis suggests we need something like a 1,000,000x scale-up of total RL compute to give a boost similar to a GPT-level jump—a bearish implication for RLVR scaling.
“Toby Ord has a great post where he cleverly connects the dots between different o-series benchmark charts, which suggested “we need something like a 1,000,000x scale-up of total RL compute to give a boost similar to a GPT level”.”
Robotics is fundamentally an algorithms problem, not a hardware or data problem, because humans can learn to teleoperate current hardware to do useful work with very little training; a human-like learner would largely solve robotics, but the absence of such a learner forces collecting data across thousands of homes to learn tasks like folding laundry.
“In some fundamental sense, robotics is an algorithms problem, not a hardware or data problem — with very little training, humans can learn how to teleoperate current hardware to do useful work. So if we had a human like learner, robotics would (in large part) be solved.”
It is contradictory to have short AGI timelines while also being bullish on the current reinforcement-learning-atop-LLMs scaleup, because if we were actually close to a human-like learner, the whole approach of training on verifiable outcomes would be doomed.
“I’m confused why some people have short timelines and at the same time are bullish on the current scale up of reinforcement learning atop LLMs. If we’re actually close to a human-like learner, this whole approach of training on verifiable outcomes is doomed.”
If capabilities were truly at AGI level, firms would spend trillions a year buying tokens since knowledge workers cumulatively earn tens of trillions in wages; the fact that lab revenues are four orders of magnitude below this proves the models are nowhere near as capable as human knowledge workers.
“If the capabilities were actually at AGI level, people would be willing to spend trillions of dollars a year buying tokens (knowledge workers cumulatively earn 10s of trillions of dollars of wages a year). The reason that lab revenue are 4 orders of magnitude off right now is that the models are nowhere near as capable as human knowledge workers.”
The claim that AI isn't widely deployed because technology takes a long time to diffuse is cope used to gloss over the fact that current models simply lack the capabilities necessary for broad economic value outside coding.
“Sometimes people will say that the reason that AIs aren’t more widely deployed across firms and already providing lots of value (outside of coding) is that technology takes a long time to diffuse. I think this is cope. People are using this cope to gloss over the fact that these models just lack the capabilities necessary for broad economic value.”
Solving continual learning won't be a singular one-and-done achievement but will resemble in-context learning: just as GPT-3 demonstrated powerful but unfinished ICL that required years of further progress, labs will release something they call continual learning next year that counts as real progress, but human-level continual learning may take another 5 to 10 years.
“GPT-3 demonstrated that in context learning could be very powerful ... But of course, we didn’t “solve” in-context learning when GPT-3 came out ... I expect a similar progression with continual learning. Labs will probably release something next year which they call continual learning ... But human level continual learning may take another 5 to 10 years of further progress.”
Every day a worker does a hundred things requiring judgment, situational awareness, and skills and context learned on the job, and these tasks differ across people and even day-to-day for the same person, so it is not possible to automate even a single job by baking in a predefined set of skills, let alone all jobs.
“Every day, you have to do a hundred things that require judgment, situational awareness, and skills & context learned on the job. These tasks differ not just across different people, but from one day to the next even for the same person. It is not possible to automate even a single job by just baking in some predefined set of skills, let alone all the jobs.”
While baking in fluency with common tools makes sense and knowledge-sharing across copies is a key AGI advantage, people underrate how much company- and context-specific skill is required for most jobs, and there is currently no robust, efficient way for AIs to pick up those skills.
“people are underrating how much company and context specific skills are required to do most jobs. And there just isn’t currently a robust efficient way for AIs to pick up those skills.”
There won't be runaway gains to the first lab that cracks continual learning, because it won't arrive as a fully-solved drop-out-of-nowhere breakthrough (which would be 'game set match'); instead a lab gets initial traction, playing with the feature reveals how it was implemented, and competitors quickly replicate and slightly improve it.
“This is why I don’t expect some kind of runaway gains to the first model that cracks continual learning ... Instead, some lab is going to figure out how to get some initial traction on the problem. Playing around with this feature will make it clear how it was implemented, and the other labs will soon replicate this breakthrough and improve it slightly.”
The takeoff-within-5-years argument that cludgy RL will build a superhuman AI researcher who then solves robust learning-from-experience is implausible, because it amounts to expecting an automated researcher to figure out the algorithm for AGI—a problem humans have struggled with for nearly a century—while itself lacking the basic learning capabilities children have.
“Somehow this automated researcher is going to figure out the algorithm for AGI - a problem humans have been banging their head against for the better part of a century - while not having the basic learning capabilities that children have? I find this super implausible.”
People underestimate how big a deal actual AGI will be because they imagine more of the current regime rather than billions of human-like intelligences on a server that can copy and merge all their learnings; the author expects this actual AGI within the next decade or two.
“They’re not thinking about billions of human-like intelligences on a server which can copy and merge all their learnings. And to be clear, I expect this (aka actual AGI) in the next decade or two. That’s fucking crazy!”
The future might feature continual-learning agents going out to do jobs and generate value, then bringing their learnings back to a hive-mind model that batch-distills across all agents; the agents could be specialized, containing Karpathy's 'cognitive core' plus job-relevant knowledge and skills.
“the future might look continual learning agents going out, doing jobs and generating value, and then bringing all their learnings back to the hive mind model, which does some kind of batch distillations on all these agents. The agents themselves could be quite specialized - containing what Karpathy called “the cognitive core” plus knowledge and skills relevant to the job they’re being deployed to do.”
The labs' own actions—pre-baking economically valuable skills like crafting PowerPoint slides—reveal a worldview in which models will continue to generalize and learn-on-the-job poorly, since you wouldn't need to pre-bake a consultant's slide-making skills if you only aimed to automate an AI researcher.
“You don’t need to pre-bake the consultant’s skills at crafting Powerpoint slides in order to automate Ilya. So clearly the labs’ actions hint at a world view where these models will continue to fare poorly at generalizing and on-the-job learning, thus making it necessary to build in the skills that they hope will be economically valuable.”
One of the key advantages AGIs will have over humans is a greater capacity to share knowledge across copies.
“Indeed one of the key advantages that AGIs will have is this greater capacity to share knowledge across copies.”