What this covers

This is a solo argument from Dwarkesh Patel about where recent AI gains actually come from. He starts from the observation that open models trail closed frontier models by only about four months. He reads that as a Catch-Up as Diagnostic of Progress Driver: data can be distilled from public APIs, while architectural tricks cannot. He then treats RL as synthetic data generation against a verifier, one that only works if the model already has some chance of producing the right answer. That requirement explains the expert-data industry. Patel cites Mercor and Surge listings for legal experts, consultants, and others, and says each skill takes at least hundreds of experts.

The middle section compares humans and models. A person takes in roughly 200 million tokens by adulthood, against trillions for frontier models, and teenagers learn to drive in about 20 hours. Patel then works through the usual rebuttals. Adding sensory data narrows the gap only partly, and deaf and blind people are still fully general. Scaling Laws under Chinchilla constants cut data needs by about a factor of 10 at most. A roughly 3GB genome cannot hold model weights, which weakens the Evolution as Pretraining Analogy. Pretrained models still need far more examples than an educated human to pick up a new programming language. He also ties the robotics shortfall to this gap.

The last stretch turns to economics and forecasts. Inefficient training can still pay, because learned skills are amortized across billions of sessions. The labs' route to out-of-distribution work is to automate AI research first and let it solve sample efficiency. Patel calls current Intelligence Explosion thinking clumsy, and bets on more demand for human software engineers in 2028.

Sharpest takeaway

Patel argues that AI progress has come from massive, task-specific expert data and RL rather than improved sample efficiency, so the gap between human and AI learning is real and is not explained away by evolution, sensory data, or scaling alone.

  • Open models catch up to frontier closed models within months because data, which is easily distilled, drives progress rather than hyperparameters or architecture
  • Humans are thousands to millions of times more sample efficient, and parameter scaling can only reduce the data needed by about a factor of 10 under Chinchilla-style scaling laws
  • Commercial value from bringing common tasks into distribution justifies inefficient training, so sample efficiency matters mainly as a route to automating AI research

The claims · ranked31 claims · weighted by value

This asset isn't compiled yet

You're seeing its claims, ranked. Compile it to build the argument threads, weight them, and check each claim against your library — the full view.

0.75

Under the Chinchilla scaling constants, where parameter and data loss terms add independently, even infinitely many parameters would reduce the data needed for a given loss by only about a factor of 10, and the result would not change with different constants, so scaling cannot close a gap of thousands to millions of times.

factualhigh valuecontestednovelty 4/4durability 3/4· Dwarkesh Patel

“even if you increased the number of parameters by infinity, that would only decrease by a factor of ~10 the amount of data you need in order to keep the same loss. Humans are somewhere between thousands to millions of times more sample efficient than these models. Scaling of current models simply can’t make up for that discrepancy.”

0.74

Open models lag closed frontier models by only about 4 months, and this fast catch-up suggests data is the real driver of progress because data can be distilled from public APIs whereas hyperparameters, training tricks, and architectural micro-optimizations cannot be easily copied; if those were the main drivers, catching up would be slower.

causalhigh valuecontestednovelty 4/4durability 2/4· Dwarkesh Patel

“Epoch recently reported that open models only lag state-of-the-art closed models by 4 months. I think the reason it is relatively easy for open source and previous laggards to catch up to within months of the frontier is that data is the real driver of progress.”

0.73

Blind and deaf people who lack a sense still have general intelligence equal to everyone else, and deaf people who communicate only via sign language and reading ingest far fewer than 200 million language tokens yet are fully general intelligences, suggesting sensory tokens are not what makes humans smart.

causalhigh valuecontestednovelty 3/4durability 3/4· Dwarkesh Patel

“In fact, deaf people who can only communicate via sign language and reading (and not from hearing) are ingesting far less than the 200 million language tokens we calculated earlier, and even this is sufficient for them to be fully general intelligences.”

0.73

For RL-based training to work, the model must already have some nonzero prior probability of producing the correct solution, which is why mind-stretching amounts of human expert trajectories are needed in every skill the model should master.

causalhigh valuecontestednovelty 3/4durability 3/4· Dwarkesh Patel

“For this process to work, the model must have at least prior some probability to anticipate the correct solution, which is why you also need mind-stretching amounts of human expert trajectories in every single field and skill you want the model to be competent at.”

0.72

Models must grind far more numerous and harder tasks than humans, since GRPO generates hundreds to thousands of rollouts per task whereas a human student might practice a textbook problem once or twice, making the process look like a Frankenstein's monster of grafted examples.

factualhigh valueestablishednovelty 3/4durability 2/4· Dwarkesh Patel

“Whereas a human student might practice a textbook problem once or twice, GRPO has the model generate hundreds to thousands of rollouts per task.”

0.71

Reinforcement learning can be understood as synthetic data generation: a large amount of compute is applied against a verifier to find good outputs, and the model is then trained to predict those correct rollouts much as it predicts next words in internet text.

definitionhigh valueestablishednovelty 2/4durability 3/4· Dwarkesh Patel

“You can think of RL as a kind of synthetic data generation - you dump a lot of compute against a verifier in order to find the “good” data.”

0.71

The main way AIs have improved is by adding more and better data and scaling the compute used to generate that data, with RL being the dominant method, which works as a kind of synthetic data generation against a verifier.

causalhigh valueestablishednovelty 2/4durability 3/4· Dwarkesh Patel

“The main way that AIs have been getting better is from addingmore and better data, and scaling the compute to develop that data in the first place.”

0.70

Human expert data for AI training is highly task-specific and bespoke, illustrated by job listings on Mercor and Surge for roles such as word specialists, legal experts writing M&A diligences or securities filings, and management consultants writing template market research.

factualhigh valueestablishednovelty 2/4durability 2/4· Dwarkesh Patel

“It’s hard to overstate how task specific and bespoke this human expert data is. If you want to get some intuition, go read some job descriptions at Mercor or Surge’s websites.”

0.69

It is not clear that AI labs have made much progress on training sample efficiency in recent years; instead, progress has come mainly from dramatically widening and improving the data distribution.

factualhigh valuecontestednovelty 3/4durability 2/4· Dwarkesh Patel

“It’s not clear that we’ve actually made much progress on training sample efficiency over the last few years - it seems like more so we’ve dramatically widened and improved the data distribution.”

0.62

A person who hears and sees about 2,000 words an hour sees roughly 200 million tokens from birth to adulthood, whereas frontier models are trained on somewhere between 10 and hundreds of trillions of tokens, a near-million-fold difference.

factualhigh valuecontestednovelty 3/4durability 2/4· Dwarkesh Patel

“If a person hears and sees on average ~2,000 words an hour, then from birth to adulthood, they’ll see ~200 millions tokens. By contrast, frontier models are trained on somewhere between 10s to 100s of trillions of tokens. That is close to a million fold difference.”

0.62

A teenager learns to drive with about 20 hours of practice, and even counting roughly 16 years of accumulated physical intuition, human driving learning uses at least 3 to 4 orders of magnitude less data than Waymo and Tesla have needed for self-driving models.

factualhigh valuecontestednovelty 3/4durability 2/4· Dwarkesh Patel

“A teenager can learn to drive a car with about 20 hours of practice. Even if you include their ~16 years of accumulated physical intuition as relevant training data, that is at least 3-4 orders of magnitude less than the amount of data Waymo and Tesla have needed”

0.59

Because humans are thousands to millions of times more sample efficient than current models and scaling cannot account for this difference, humans appear to lie on a different scaling curve altogether.

causalhigh valuespeaker onlynovelty 4/4durability 3/4· Dwarkesh Patel

“This really does suggest that humans are on a different scaling curve altogether.”

0.57

The objection that billions of years of evolution count as pretraining fails because the human genome is about 3GB with 1 to 2 percent protein coding, far too small to store the model parameters of frontier models, so evolution more plausibly found good hyperparameters and loss functions while parameter learning still happens within a lifetime.

causalhigh valuespeaker onlynovelty 3/4durability 3/4· Dwarkesh Patel

“Our genome is 3GB, about 1-2% protein coding. That is just not enough space to store the model parameters that are supposedly pretrained (frontier models are terabytes sized).”

0.57

Training AIs is far less efficient than training humans, but this does not matter economically because AIs can learn skills by firehosing gigawatts of compute and their learned skills can be amortized across billions of sessions, so inefficiency is still wildly profitable.

normativehigh valuespeaker onlynovelty 3/4durability 3/4· Dwarkesh Patel

“But AIs can learn these skills by firehosing gigawatts of training at a time. And what they learn can be amortized across billions of sessions, so we can be ludicrously inefficient in training them and still be wildly in the green.”

0.57

A person can learn to teleoperate any humanoid or robot arm within hours, and robotics has not become a multi-trillion dollar industry because AIs learn far less efficiently than humans, so even millions of hours of demonstrations are insufficient for complex open-ended tasks.

causalhigh valuespeaker onlynovelty 3/4durability 3/4· Dwarkesh Patel

“A person can learn to teleoperate any random humanoid or robot arm within hours. The reason robotics isn’t already a deca-trillion dollar industry... is that our AIs learn so much less efficiently than humans”

0.56

Scaling laws suggest bigger models are more sample efficient, and if the human brain's roughly 100 trillion synapses map to parameters, another order of magnitude or two of scaling could reach human-level efficiency, though Patel later argues this route is insufficient.

forecasthigh valuecontestednovelty 2/4durability 2/4· Dwarkesh Patel

“Scaling laws tell us that bigger models are more sample efficient. The human brain is 100T synapses - if each synapse is ~1 parameter, and frontier models are currently roughly ~5T parameters, then maybe we could achieve human-level sample efficiency with another order of magnitude or two of parameter scaling.”

0.52

Even if pretraining on trillions of tokens can be explained as catching up with evolution, it does not explain why marginal capabilities require so much data, since an educated human does not need 100 different professors to learn a new programming language while pretrained AIs still do.

causalhigh valuespeaker onlynovelty 3/4durability 3/4· Dwarkesh Patel

“once you have been educated, you don’t need 100 different professors to learn a new programming language, but the AIs (even once pretrained) do.”

0.50

Patel bets there will be more demand for human software engineers in 2028 than now, largely due to the complementary input of AI, since software engineering involves substantial out-of-distribution problems.

forecasthigh valuespeaker onlynovelty 3/4durability 1/4· Dwarkesh Patel

“I would be willing to bet that there’s overall more demand for human software engineers in 2028 than there is now, largely due to the complementary input of AI.”

0.50

Human lifespan does not allow the quantity and breadth of training that AI models receive; a human who needed to read every public GitHub repository before becoming competent would be unemployable, while an AI can learn these skills.

causalhigh valuespeaker onlynovelty 2/4durability 3/4· Dwarkesh Patel

“If you as a human had some weird learning disability where you needed to read through every public repository on Github before you could be a competent developer, it would not make sense to train you up.”

0.49

Current thinking about an intelligence explosion is clumsy because people either dismiss AIs speeding up AI progress altogether or simply assume a superintelligence pops out the other end, rather than reasoning concretely about what extremely rapid progress starting from LLMs would look like.

normativehigh valuespeaker onlynovelty 2/4durability 2/4· Dwarkesh Patel

“I think the way that people currently think about an intelligence explosion is pretty clumsy. Either people dismiss the possibility of AIs speeding up AI progress altogether, or they just assume that God pops out the other end.”

0.49

Each AI skill corresponds to at least hundreds of human experts generating example completions, writing rubrics, and explaining their chain of thought, which is why the data industry producing these labels earns billions a year and soon will earn deca-billions.

factualhigh valuespeaker onlynovelty 2/4durability 2/4· Dwarkesh Patel

“Each skill corresponds to at least hundreds of human experts who are generating example completions, writing rubrics, and explaining their chain of thought.”

0.49

The labs' strategy for out-of-distribution white-collar work is to first automate AI research, then have automated AI researchers solve the sample efficiency problem, so the key question becomes whether AIs lacking human-level sample efficiency can still solve the research problems on the way to human-like learning.

definitionhigh valuespeaker onlynovelty 2/4durability 2/4· Dwarkesh Patel

“The labs’ plan for these later kinds of jobs is to first automate AI research, and then have the automated AI researchers solve this sample efficiency problem.”

0.49

The bet on white-collar work is that common tasks done by software engineers, analysts, and accountants can be brought into distribution easily through RL and SFT, and AI lab revenue curves suggest large value comes from this even without human-level sample efficiency.

forecasthigh valuespeaker onlynovelty 2/4durability 2/4· Dwarkesh Patel

“The bet with white collar work is the common tasks that a software engineer or analyst or accountant does are, well common. And we can bring common tasks into distribution quite easily through RL and SFT.”

0.36

Including multimodal sensory information in the comparison brings the human total from birth to adulthood to roughly 10 to 100 billion tokens, which still leaves humans far below frontier model training scale.

factualcontestednovelty 2/4durability 2/4· Dwarkesh Patel

“If you include all this sensory information, we’re probably in the 10s to 100s of billions of tokens range from birth to adulthood”

0.30

Adam Marblestone argued in a prior podcast that the loss functions evolution discovered were the more significant find than any encoded knowledge.

factualspeaker onlynovelty 3/4durability 2/4· Dwarkesh Patel

“I had an interesting podcast with Adam Marblestone where he argued that the loss functions were the more significant find from evolution”

0.29

One definition of intelligence is sample efficiency, meaning how much data one needs to see in a given domain in order to operate fluently and competently.

definitionspeaker onlynovelty 2/4durability 3/4· Dwarkesh Patel

“One definition of intelligence is sample efficiency - that is to say, how much data do you need to see in a given domain in order to operate fluently and competently.”

0.26

Current training is building a Frankenstein's monster, a system assembled from a billion grafts of carefully constructed examples sewn together, rather than a unified learner.

normativespeaker onlynovelty 2/4durability 2/4· Dwarkesh Patel

“We are building some Frankenstein’s monster, with a billion grafts of carefully constructed examples sewn together.”

0.26

How much out-of-distribution thinking white-collar jobs require depends on the nature of the job, since some roles like bank tellers and travel agents were automated long before modern AI, while others such as software engineering involve problems far from the training distribution.

factualspeaker onlynovelty 2/4durability 2/4· Dwarkesh Patel

“some jobs are mechanical and predictable enough that they were automated long before the modern era of AI, for example bank tellers or travel agents. And there are other jobs which require dealing on a daily basis with problems that are quite distant from the data distribution.”

0.26

Although AI capabilities look like a galaxy of glittering abilities, the core that holds them together is an enormous black hole of data, since frontier models see vastly more data than humans encounter in a lifetime.

factualspeaker onlynovelty 2/4durability 2/4· Dwarkesh Patel

“We see these AIs as a galaxy glittering with capabilities, but at their center, invisible to the naked eye, holding all the constellations together, is an unimaginably massive black hole of data.”

0.13

Patel says he will address how an intelligence explosion would unfold in a future blog post.

forecastspeaker onlynovelty 0/4durability 1/4· Dwarkesh Patel

“That question I’ll address in a future blog post”

0.13

Mercury is a banking platform whose new AI feature Command can send invoices, categorize expenses, and transfer money.

factualspeaker onlynovelty 0/4durability 1/4· Dwarkesh Patel

“Mercury is my banking platform, and they just released a new AI feature called Command.”