What this covers

Andrej Karpathy and Dwarkesh Patel discuss the feasibility and timeline of AGI, with Karpathy arguing that large language models are fundamentally limited imitations of human-generated internet text rather than evolved systems, and that despite real progress over the past three decades, these models have deep cognitive deficits that will extend the path to artificial general intelligence by roughly a decade. The conversation spans the nature of current AI systems, why their practical deployment remains harder than their capabilities suggest, and how transformative automation will likely integrate into the existing economic growth trajectory rather than causing a discontinuous explosion.

The discussion ranges across multiple domains: the history of algorithmic progress in machine learning, how self-driving's "march of nines" reveals vast gaps between demonstration and production safety, the difference between surface and deep understanding in learning, and how reinforcement learning remains a crude mechanism for credit assignment. Karpathy contrasts LLMs' current lack of continual learning, culture, and self-play mechanisms with the capabilities that would be needed for true AGI, and argues that AI as Continuation of Computing means it will diffuse into economies at historical rates rather than breaking the ~2% growth exponential. The conversation also dwells extensively on education and knowledge transfer—what distinguishes memorization from understanding, how tutoring works, why experts struggle to teach, and the pedagogical superiority of spoken explanation over written exposition. Throughout, Karpathy elaborates on what remains unclaimed research territory and why the field, despite its achievements, is still genuinely early.

Sharpest takeaway

Karpathy argues that today's LLMs are 'ghosts' built by imitation rather than evolved animals, and that despite genuine progress they have deep cognitive deficits—so AGI will take roughly a decade and will diffuse into the existing ~2% GDP growth trend rather than triggering a discontinuous explosion.

  • LLMs lack continual learning, true culture, self-play, and entropy-rich generation; RL is a crude credit-assignment mechanism
  • Self-driving's decade-long 'march of nines' shows demo-to-product gaps in safety-critical domains, including software
  • AI is continuous with prior computing automation, which historically never broke the 2% growth exponential

The claims · ranked46 claims · weighted by value

This asset isn't compiled yet

You're seeing its claims, ranked. Compile it to build the argument threads, weight them, and check each claim against your library — the full view.

0.83

Reproducing LeCun's 1989 net showed that 33 years of algorithmic progress alone only halved the error; real gains required simultaneously improving data (10x the training set), compute, kernels, and algorithms—no single factor dominates, each contributes roughly equally, so future progress will likewise come from broad-based ~20% improvements across the stack.

causalhigh valueestablishednovelty 3/4durability 4/4· Andrej Karpathy

“if I time travel by algorithms 33 years, I could adjust what Yann LeCun did in 1989, and I could halve the error. But to get further gains, I had to add a lot more data”

0.80

There are two types of knowledge—shallow surface knowledge and deep build-it knowledge—and building something from scratch (without copy-paste) forces you to confront what you don't understand and don't know that you don't understand, embodying Feynman's principle that if you can't build it you don't understand it.

normativehigh valueestablishednovelty 2/4durability 4/4· Andrej Karpathy

“If I can't build it, I don't understand it. That's a Feynman quote, I believe.”

0.80

Effective learning alternates between depth-wise, on-demand learning—pursuing a project that gives a reward and learning what's needed for it—and breadth-wise learning where you trust you'll need foundations later; on-demand learning is especially motivating, and re-explaining things to others fills gaps in your own understanding.

normativehigh valueestablishednovelty 2/4durability 4/4· Andrej Karpathy

“you need a bit of alternation of learning depth-wise, on demand... and learning breadth-wise”

0.80

An excellent one-on-one tutor instantly probes and understands a student's world model—what they know and don't—and serves material precisely calibrated to their current capability so they are never bored by the trivial or overwhelmed by the too-hard, making the student the only constraint on learning; no LLM can do this yet.

factualhigh valueestablishednovelty 2/4durability 4/4· Andrej Karpathy

“Instantly from a very short conversation, she understood where I am as a student, what I know and don't know.”

0.80

Good teaching presents the problem or 'pain' before the solution and prompts students to attempt it themselves first, because guessing builds an appreciation of the action space and objective, so that when the solution arrives the student understands why each piece is necessary—maximizing knowledge gained per new fact added.

normativehigh valueestablishednovelty 2/4durability 4/4· Andrej Karpathy

“You're presenting the pain before you present a solution, and how clever is that?”

0.80

Experts are often bad at explaining their field due to the curse of knowledge—they take things for granted and cannot put themselves in beginners' shoes; one remedy is having beginners share their dumb questions (e.g. ChatGPT conversations) so the expert can recover the novice perspective and explain better.

causalhigh valueestablishednovelty 2/4durability 4/4· Andrej Karpathy

“It's the curse of knowledge and expertise. This is a real phenomenon... you take certain things for granted, and you can't put yourself in the shoes of new people”

0.80

How an author would explain their paper conversationally over lunch or beers is consistently more understandable and even more accurate and scientific than the written version, because writing biases people toward abstract, jargon-filled exposition and throat-clearing, whereas one-on-one communication compels them to just say the thing.

factualhigh valueestablishednovelty 2/4durability 4/4· Andrej Karpathy

“people have a bias to explain things in the most abstract, jargon-filled way possible and to clear their throat for four paragraphs before they explain the central idea”

0.78

Self-driving progress is a 'march of nines' where each additional nine of reliability costs roughly a constant amount of work; a 90% demo is just the first nine, and the demo-to-product gap is huge in safety-critical domains where failure cost is high—including production software, where a mistake can be unboundedly bad.

definitionhigh valuecontestednovelty 3/4durability 4/4· Andrej Karpathy

“it's a march of nines. Every single nine is a constant amount of work.”

0.77

AI will not instantly replace people; instead an autonomy slider will swap in AIs that do ~80% of the volume in amenable jobs like call centers (simple, repetitive, low-context, purely digital), delegating ~20% to human supervisors managing teams of AIs, requiring new interface companies to manage imperfect AIs.

forecasthigh valuecontestednovelty 3/4durability 3/4· Andrej Karpathy

“We're going to be swapping in AIs that do 80% of the volume. They delegate 20% of the volume to humans, and humans are supervising teams of five AIs”

0.77

LLMs lack two powerful multi-agent ideas: culture (a growing repertoire of knowledge they write down and share, like one LLM writing a book for others) and self-play (LLMs generating progressively harder problems for each other to solve, as AlphaGo did against itself); both remain unclaimed research realms, which is one reason the field is still early.

factualhigh valuecontestednovelty 3/4durability 3/4· Andrej Karpathy

“LLMs don't really have culture right now and it's one of the impediments I would say.”

0.77

AI is continuous with prior computing automation and will not produce a discontinuous growth jump; like computers, mobile phones, and the iPhone, transformative technologies are not visible as breaks in GDP because they diffuse so slowly that they average into the same ~2% exponential, and AI—still fundamentally a program—will do the same while eventually making the curve extremely vertical.

forecasthigh valuecontestednovelty 3/4durability 3/4· Andrej Karpathy

“I was trying to find AI in the GDP for a while. I thought that GDP should go up. But then I looked at some of the other technologies... You can't find them in GDP. GDP is the same exponential.”

0.75

Geoff Hinton's prediction that radiologists would no longer be a job was very wrong; radiologists are alive, well, and growing even though computer vision is excellent at image recognition, because the job is messy and complicated with many surfaces and patient interaction.

factualhigh valueestablishednovelty 2/4durability 3/4· Andrej Karpathy

“Geoff Hinton's prediction that radiologists would not be a job anymore, and this turned out to be very wrong”

0.75

Software automation should diffuse far faster than self-driving because bits are roughly a million times easier than anything touching the physical world—arbitrarily reshufflable at rapid speed—and the economics are favorable since serving another model instance costs far less than building another car, though latency and compute-serving constraints remain.

causalhigh valueestablishednovelty 2/4durability 3/4· Andrej Karpathy

“If you're sticking to the realm of bits, bits are a million times easier than anything that touches the physical world.”

0.73

There is no clean line where AI begins; it is a continuum extending the recursive self-improvement of computing—from code editors, syntax highlighting, and type checking to search engine ranking and now autocomplete and agents—where the 'autonomy slider' moves humans progressively up the abstraction stack as more is automated.

definitionhigh valuecontestednovelty 3/4durability 3/4· Andrej Karpathy

“I see AI as fundamentally an extension of computing in a pretty fundamental way.”

0.73

Process-based supervision via LLM judges fails because the judges have billions of parameters and are gameable: RL against them quickly finds adversarial examples (e.g. nonsensical 'dhdhdhdh' completions that get 100% reward), since these are out-of-sample and lie in pure generalization land, and patching each one only yields a new model with an infinity of new adversarial examples.

causalhigh valuecontestednovelty 3/4durability 3/4· Andrej Karpathy

“those LLMs are giant things with billions of parameters, and they're gameable. If you're reinforcement learning with respect to them, you will find adversarial examples for your LLM judges, almost guaranteed.”

0.73

We are not building animals shaped by evolution with hardware baked in; we are building 'ghosts' or spirits—fully digital entities created by imitating human internet data—so analogies to animal learning are misleading and pre-training is a 'crappy evolution' that gives a starting point of knowledge and intelligence.

definitionhigh valuecontestednovelty 3/4durability 3/4· Andrej Karpathy

“we're not building animals. We're building ghosts or spirits or whatever people want to call it, because we're not doing training by evolution”

0.73

True AGI would be qualitatively different from prior productivity technologies because it is labor itself; we live in a labor-constrained world, and billions of extra human-like minds inventing things and integrating themselves into the economy—like adding 10 billion people or a Hong Kong/Shenzhen-style catch-up—could shift growth to a 10-20% regime, a discontinuity comparable to the Industrial Revolution's jump from 0.2% to 2%.

forecasthigh valuecontestednovelty 3/4durability 3/4· Dwarkesh Patel

“true AGI... is qualitatively different from these other productivity-improving technologies because it's labor itself”

0.73

Humans are bad at memorization, which is a feature not a bug, because it forces them to find generalizable patterns and see the forest for the trees; LLMs are too good at memorization (they can regurgitate random hashed text after one or two iterations), which distracts them with pre-training documents, so the cognitive core should deliberately remove memory.

causalhigh valuecontestednovelty 3/4durability 3/4· Andrej Karpathy

“Because we're not that good at memorization, we're forced to find patterns in a more general sense.”

0.73

Coding is the ideal first domain for LLMs because it has always been text-based, LLMs are perfect text processors with abundant code data, and there is pre-built infrastructure (IDEs, diffs) for handling and reviewing code; domains like slides lack this because they are spatial visual graphics with no diff infrastructure.

causalhigh valuecontestednovelty 3/4durability 3/4· Andrej Karpathy

“coding has always fundamentally worked around text... LLMs, the way they're trained on the Internet, love text. They're perfect text processors”

0.73

Early attempts at full agents—Atari deep RL in 2013 and OpenAI's Universe project using keyboard/mouse on web pages—were missteps because they were far too early: without the representation power of a pre-trained language model, rewards are too sparse and the agent just keyboard-mashes and burns compute without learning.

factualhigh valuecontestednovelty 3/4durability 3/4· Andrej Karpathy

“this was extremely early, way too early, so early that we shouldn't have been working on that”

0.71

InstructGPT was mind-blowing because it showed that an autocomplete base model could be fine-tuned on conversation-like text and rapidly adapt to become a conversational assistant while retaining all pre-training knowledge, demonstrating that stylistic adaptation requires only a few loops of fine-tuning.

factualhigh valueestablishednovelty 2/4durability 3/4· Andrej Karpathy

“if you just fine-tune it on text that looks like conversations, the model will very rapidly adapt to become very conversational, and it keeps all the knowledge from pre-training”

0.71

The most likely outcome of advanced automation is a gradual loss of both understanding and control, as we layer AI everywhere with fewer people understanding it, plausibly resulting in multiple competing autonomous entities—some going rogue while others fight them off—acting on behalf of people but producing outcomes society doesn't want.

forecasthigh valuecontestednovelty 2/4durability 3/4· Andrej Karpathy

“there will be a gradual loss of control and understanding of what's happening. That to me seems the most likely outcome”

0.70

Labs are being practical with fixed flops and cost budgets: pre-training is not where you want most of your flops, so models have gotten smaller in the pre-training stage but make it up in reinforcement learning and mid-training to get the most bang for the buck.

factualhigh valueestablishednovelty 2/4durability 2/4· Andrej Karpathy

“It just turns out that pre-training is not where you want to put most of your flops or your cost. That's why the models have gotten smaller.”

0.70

Reinforcement learning is terrible because after a long rollout you get a single final reward bit, and every token along a successful trajectory gets upweighted equally—even the wrong alleys—broadcasting one noisy supervision signal across the whole trajectory; humans instead do far fewer attempts plus a rich review of which parts went well, which current LLMs have no equivalent of.

causalhigh valuecontestednovelty 3/4durability 3/4· Andrej Karpathy

“you're sucking supervision through a straw. You've done all this work that could be a minute of rollout, and you're sucking the bits of supervision of the final reward signal through a straw”

0.70

You cannot just generate synthetic reflections and train on them because model outputs are silently collapsed—any individual sample looks great but the distribution occupies a tiny manifold (e.g. ChatGPT only knows about three jokes), lacking human entropy and diversity; training on this collapsed data makes models worse, making entropy-preserving synthetic generation an unsolved research problem.

causalhigh valuecontestednovelty 3/4durability 3/4· Andrej Karpathy

“all of the samples you get from models are silently collapsed. Silently—it is not obvious if you look at any individual example of it—they occupy a very tiny manifold of the possible space”

0.70

Intelligence evolves only on a narrow line: it must be too important and variable to bake directly into DNA weights yet important enough to learn at all; unpredictable, rapidly changing environments that evolution can't pre-bake incentivize building a lifetime learning algorithm, and humans hit a niche (hands, tool use, externalized digestion) rewarding marginal intelligence with a scalable brain.

causalhigh valuecontestednovelty 3/4durability 3/4· Andrej Karpathy

“You want environments that are unpredictable so evolution can't bake your algorithms into your weights.”

0.69

Most animal reinforcement learning is for motor tasks, not intelligence tasks; a lot of what looks like learning is actually maturation, so humans largely do not use reinforcement learning for problem-solving and high-level cognition.

causalhigh valuecontestednovelty 3/4durability 2/4· Andrej Karpathy

“I actually kind of think humans don't really use RL, roughly speaking.”

0.69

Coding agents are good at boilerplate and code common on the internet, but they fail on unique, intellectually intense code: they over-rely on memorized common patterns, bloat code with defensive try-catch and production scaffolding, use deprecated APIs, and can't internalize custom non-standard implementations—so for novel work autocomplete is the sweet spot rather than full agents.

factualhigh valuecontestednovelty 3/4durability 2/4· Andrej Karpathy

“They're not very good at code that has never been written before, maybe it's one way to put it”

0.69

Self-driving is nowhere near done: deployments are minimal and uneconomical, the cars rely on elaborate teleoperation centers with more human-in-the-loop than people realize, so rather than removing the human we have moved them somewhere out of sight.

factualhigh valuecontestednovelty 3/4durability 2/4· Andrej Karpathy

“there are very elaborate teleoperation centers of people kind of in a loop with these cars”

0.69

Today's models cognitively resemble elementary or kindergarten students—savant kids with perfect memory who can pass PhD quizzes and produce convincing slop, but who don't really know what they're doing and lack cognition across many checkboxes—so they can't yet create culture because they're still kids.

factualhigh valuecontestednovelty 3/4durability 2/4· Andrej Karpathy

“They're savant kids. They have perfect memory of all this stuff. They can convincingly create all kinds of slop that looks really good.”

0.68

The original OpenAI definition of AGI was a system that can do any economically valuable task at human performance or better; people now quietly concede away all physical work, leaving digital knowledge work that is maybe 10-20% of the economy—still a multi-trillion-dollar market.

definitionhigh valuecontestednovelty 2/4durability 3/4· Andrej Karpathy

“AGI was a system you could go to that can do any economically valuable task at human performance or better.”

0.66

Knowledge stored in the weights is only a 'hazy recollection' of training data due to massive compression (15 trillion tokens into a few billion parameters), whereas tokens in the context window are directly accessible like working memory—which is why feeding a model a full chapter yields far better answers than relying on its trained memory.

definitionhigh valuecontestednovelty 3/4durability 3/4· Andrej Karpathy

“the knowledge is only a hazy recollection of what happened in training time”

0.66

In-context learning is pattern completion within a token window that emerges from pre-training, and there is evidence (e.g. a linear-regression-in-context paper) that it may internally run a small gradient-descent-like loop in the layers of the network rather than being a fundamentally different mechanism.

factualhigh valuecontestednovelty 3/4durability 3/4· Andrej Karpathy

“I do think it's possible that in-context learning runs a small gradient descent loop internally in the layers of the neural network.”

0.63

Current agents like Claude and Codex are impressive but still don't work as reliable employees because they lack intelligence, multimodality, computer use, and continual learning; working through all these deficits will take roughly a decade, not a year.

forecasthigh valuecontestednovelty 2/4durability 2/4· Andrej Karpathy

“this is more accurately described as the decade of agents”

0.63

Much of the AI hype is fundraising and attention-to-money conversion, but the compute buildout is likely not overbuilding—analogous to railroads or the late-90s telecom pre-paving of the internet—because miraculous demand (Claude Code, Codex didn't exist a year ago, plus ChatGPT) will gobble up the capacity, even though some fast timelines are repeatedly wrong.

forecasthigh valuecontestednovelty 2/4durability 2/4· Andrej Karpathy

“Pre-paving the internet that only came a decade later and creating a whole bubble in the telecommunications industry in the late '90s.”

0.61

The evolution of intelligence is intuitively surprising and likely rare: bacteria stayed stuck for two billion years before eukaryotes, suggesting that step is hard, but multicellular animals and intelligence arose relatively quickly afterward; the independent emergence of distinct bird and hominid intelligence hints intelligence may spring up a few times given the right niche.

factualhigh valuecontestednovelty 2/4durability 3/4· Andrej Karpathy

“If bacteria were around for 2 billion years and nothing happened, then going to eukaryote is probably pretty hard”

0.60

Early education should emphasize physics not to accumulate knowledge but to boot up the brain with generalizable cognitive tools: building models and abstractions, first-order approximations plus higher-order correction terms, extracting fundamental frequencies from noise, and approximating complex systems (the 'spherical cow') in ways transferable across domains.

normativehigh valuespeaker onlynovelty 3/4durability 4/4· Andrej Karpathy

“early school education is not about accumulating knowledge or memory for tasks later in the industry. It's about booting up a brain. Physics uniquely boots up the brain the best”

0.59

Pre-AGI education is useful because people learn to make money, but post-AGI when everything is automated, education becomes fun and a form of self-betterment—like going to the gym for fitness rather than to manipulate heavy objects—so people will learn for psychological and evolutionary reasons even when it has no economic necessity.

forecasthigh valuespeaker onlynovelty 3/4durability 3/4· Andrej Karpathy

“pre-AGI education is useful. Post-AGI education is fun.”

0.59

Next-token prediction over the internet does two unrelated things: it absorbs knowledge and it becomes intelligent by booting up circuits like in-context learning; the knowledge is actually a liability because it makes models over-rely on the data manifold, so the goal should be to strip knowledge and keep a 'cognitive core' of pure algorithms of thought.

causalhigh valuespeaker onlynovelty 4/4durability 3/4· Andrej Karpathy

“Number one, it's picking up all this knowledge, as I call it. Number two, it's actually becoming intelligent.”

0.57

Humans also collapse over the course of their lives: children haven't overfit yet and say shocking, original things, but adults revisit the same thoughts, say more of the same, have declining learning rates, and increasingly deteriorate—suggesting entropy-seeking (talking to others, possibly dreaming) is needed to counteract collapse.

causalhigh valuespeaker onlynovelty 3/4durability 3/4· Andrej Karpathy

“Humans collapse during the course of their lives. This is why children, they haven't overfit yet.”

0.55

Education is a hard technical problem of building 'ramps to knowledge'—untangling a web of understanding and laying it out so each piece depends only on what came before, maximizing 'eurekas per second'—exemplified by micrograd (100 lines capturing all neural-net learning as recursive chain rule, everything else being mere efficiency).

definitionhigh valuespeaker onlynovelty 3/4durability 4/4· Andrej Karpathy

“education is the very difficult technical process of building ramps to knowledge”

0.55

Because the internet training data is mostly garbage, models must be huge mainly to do memory work rather than cognitive work; if intelligent models refine pre-training data down to its cognitive components and distill it, a cognitive core of around a billion parameters could hold a productive, human-like conversation while looking up facts it doesn't know.

forecasthigh valuespeaker onlynovelty 4/4durability 2/4· Andrej Karpathy

“I almost feel like we can get cognitive cores that are very good at even a billion parameters.”

0.53

When a job is 99% automated, the remaining 1% that humans must do can become a non-fungible bottleneck, so the wages of those specially-trained humans could rise tremendously—until that last 1% is automated, at which point wages fall sharply.

forecasthigh valuespeaker onlynovelty 3/4durability 2/4· Dwarkesh Patel

“that last 1% the human has to do is incredibly valuable because it's bottlenecking everything else”

0.53

Transformers resemble cortical tissue (highly plastic and general), reasoning traces resemble the prefrontal cortex, and RL fine-tuning resembles the basal ganglia—but many ancient brain structures like the hippocampus and amygdala have no LLM analog yet, which is one way to see why models still have cognitive deficits.

factualhigh valuespeaker onlynovelty 3/4durability 2/4· Andrej Karpathy

“I think that this is cortical tissue. I think when we're doing reasoning and planning inside the neural networks... that's kind of like the prefrontal cortex.”

0.40

Waymo got an early start because it could package many sensors, but Tesla took a more scalable approach, and the future of self-driving will look more like Tesla's strategy, though it still has to play out.

forecastcontestednovelty 2/4durability 2/4· Andrej Karpathy

“Waymo had an early start because you can package up so many sensors. But I do think Tesla is taking the more scalable strategy”

0.10

This is meta-commentary thanking the host and closing the conversation.

factualspeaker onlynovelty 0/4durability 0/4· Dwarkesh Patel

“That's an excellent note to close on. Andrej, that was great.”