
Skill Issue: Andrej Karpathy on Code Agents, AutoResearch, and the Loopy Era of AI
What this covers
This is a conversation between Sarah Guo and Andrej Karpathy on what working with coding agents looks like after a capability jump he dates to December. He describes his own workflow flipping from mostly hand-written code to almost none, and he treats remaining failures as problems of the operator's skill at orchestration. He argues that leverage comes from taking yourself out of the loop and keeping agents running to maximize token throughput. Examples include Dobby, a home-control agent reached over WhatsApp that replaced six bespoke apps, including Sonos, and his auto-research project, which closes the loop on training experiments. He also gives the limits. Auto-research works only where results can be measured objectively, and current models are jagged, strong on verifiable, reinforcement-trained tasks and weak on soft ones like jokes.
The middle of the discussion moves to structure and economics. Karpathy proposes encoding research organizations as editable markdown specifications that an LLM can optimize and compare. He argues for speciating models into specialized ones instead of building monolithic generalists. On work, he applies Jevons Paradox to software and cites the ATM and bank teller story as a case where displacement did not occur. He still expects professions built on digital information to change fundamentally, and he says the effect on job counts is uncertain and contested.
The later sections take up open versus closed models. He puts open-source models at roughly six to eight months behind the frontier, down from eighteen, and predicts a Linux-like equilibrium in which closed models lead while open ones serve most uses. He argues that centralization has a poor historical record, and he questions the position frontier labs are in, as well as how much sway researchers inside them really have. The conversation closes on physical robotics, which he expects to lag digital change, on the security risks of giving agents access to personal accounts, and on education, where teachers instruct agents and agents tailor the explanations. He also offers a speculative, fringe-flavored forecast that society will reorganize around the needs of agentic systems.
Karpathy argues that AI agents have undergone a fundamental capability shift enabling a new paradigm where individual engineers can delegate complex tasks to autonomous systems, fundamentally restructuring how software development, research, and knowledge work happen—moving from human-in-the-loop workflows to human-outside-the-loop automation.
- Since December, agent capabilities jumped such that Karpathy has stopped writing code by hand and now delegates most tasks to multiple parallel agents, a 80/20 → 20/80 or greater inversion of effort
- The key constraint is no longer capability but human skill at prompt engineering and system design; everything else is a 'skill issue' solvable by better agent orchestration
- This shift extends beyond software engineering to research automation (auto-research), home automation (Dobby), and knowledge work, implying a broader restructuring of how humans interact with intelligence systems
This asset isn't compiled yet
You're seeing its claims, ranked. Compile it to build the argument threads, weight them, and check each claim against your library — the full view.
The main difference between open-source and closed models will be capital intensity: open-source can match closed-model capabilities eventually, but frontier labs have orders-of-magnitude more compute capital, making it harder to compete on training large models.
“The big difference is that everything is capital uh there's a lot of capex that goes into this. Um so I think that's where things like fall apart a little bit, make it a bit harder to to compete in certain senses.”
Frontier labs face a structural misalignment: they have financial incentive to build powerful AI systems, yet they acknowledge these systems will dramatically reshape humanity and society, putting them in a position of having conflicted interests in how to develop the technology.
“you're you have a huge amount of financial incentive to uh with these frontier labs. And by your own admission, the uh the AIs are going to like really change humanity and society in very dramatic ways. And here you are basically like building the technology and benefiting from it like it and being like very allied to it through financial means. Like this was the conundrum that was in at the heart of you know, how OpenAI was started in the beginning.”
Auto-research only works for problems with objective metrics that are easy to evaluate; it fails for subjective optimization goals where you can't automatically verify if a solution is good.
“this is extremely well suited to anything that has objective metrics that are easy to evaluate. So for example, like writing kernels for more efficient CUDA, you know, code for various parts of the model, etc. are a perfect fit... So a lot of things like like are perfect fit for auto research, but many things will not be. And so they it's just if you can't evaluate then you can't auto research it, right?”
Current LLM agents have severe jaggedness: they excel at verifiable, RL-trained tasks (code generation, tool use) but fail catastrophically at soft/unverifiable tasks (like telling jokes), because they're only optimized along dimensions where rewards can be computed, leaving trained-off domains untouched.
“if today if you go to like state-of-the-art model, ChatGPT and you ask it tell me a joke, um do you know what joke you're going to get? There's the joke... The joke that apparently all the LLMs like love the most is why do scientists not trust atoms? Okay. Because they make everything up... So this is the joke you would get like three or four years ago and this is the joke you still get today.”
Centralization of AI intelligence in a few closed labs has poor historical precedent; political and economic history shows centralization systems have bad track records, so maintaining a decentralized alternative (open-source) is important for systemic resilience.
“I'm a little bit hesitant of having um I don't actually think it's like structurally I think there's some systemic risk attached to just having intelligence that are closed and that's like that's it. And I think that that's a, you know, centralization has a very poor track record in my view uh in in the past”
Researchers inside frontier labs have less sway over organizational decisions than outsiders might assume; they can contribute ideas but ultimately aren't in control of the entity's direction, especially when stakes get high.
“ultimately in the end of the day like when the stakes are really high, etc. If you're an employee at an organization, I don't actually know how much sway you're going to have on your organization what it's going to do. Like fundamentally at the end of the day um uh it's uh you're not like really in charge.”
Professions that manipulate digital information (software engineering, data analysis, writing, design) will change fundamentally due to AI, though whether they grow or shrink depends on demand elasticity and many other factors.
“things will change in these professions because of these new tools... It doesn't mean that there's going to be less of those jobs or more of those jobs because it does has to do with like demand elasticity and many other factors.”
Jevons Paradox applies to software: making software cheaper and easier to create via AI will increase demand for software, leading to more software engineering jobs, at least locally in the near term.
“the demand for software actually goes up. It's cheaper and there's more... The classical example of this always is the ATMs and the bank tellers... But basically it's just Jevons paradox. Like something becomes cheaper, so there's a lot of unlocked demand for it.”
Security and privacy concerns around giving agents full access to email, calendar, and personal digital infrastructure remain significant enough to warrant caution, even though agent capabilities are otherwise advancing rapidly.
“I didn't really take advantage of a lot of like email and calendar and all this other stuff and I didn't really have access cuz I'm still a little bit like suspicious and it's still very new and rough around the edges. So I didn't want to give it like full access to my digital life yet and part of it is just the security, privacy and uh, just being very cautious in that in that realm.”
The ATM and bank teller example is the canonical case of technological displacement that didn't happen; ATMs reduced branch operating costs, which increased the number of branches, which increased the number of tellers employed despite automation.
“The canonical example of this always is the ATMs and the bank tellers uh because there was a lot of like fear that um ATMs and computers basically uh would displace tellers. But what happened is they made like the cost of operation of of a bank branch much cheaper. And so there are more bank branches, so there are more tellers.”
Research organizations should be encoded as optimizable markdown specifications (program_md files) that describe roles, processes, and workflows, enabling comparison of organizational structures on the same hardware and automated discovery of more effective research processes through LLM optimization.
“A research organization is a set of markdown files that describe all the roles and how the whole thing connects.”
Society will collectively reshape itself to serve the needs of agentic systems; humans will serve as both sensors (gathering information) and actuators (executing tasks) for superintelligent systems, as described in the book 'Daemon'.
“humans are kind of like it's actuators, but humans are also like its sensors. Um and so, I think like collectively like society will kind of like reshape in a certain way in uh to to serve that kind of a that will kind of like end up happening collectively across the industry.”
The current software ecosystem with thousands of bespoke apps for smart home devices, treadmills, and other IoT devices is an inefficient overproduction that agents will render obsolete by directly calling exposed APIs, with agents serving as the intelligent glue layer instead.
“there's this sense that these apps that are on the app store for using these smart home devices, etc. Uh, these shouldn't even exist kind of in a certain sense. Like shouldn't it just be APIs and shouldn't agents be just using it directly?”
The physical world (atoms) will lag behind the digital world (bits) by a significant amount of time in AI-driven transformation because manipulating atoms is orders of magnitude harder than manipulating bits.
“flipping bits and and the ability to copy-paste digital information is like makes everything a million times faster than accelerating matter, you know, so... we're going to see a huge amount of activity in the digital space, huge amount of rewriting, huge amount of activity, boiling soup. And I think the we're going to see something that in the digital space goes at the speed of light compared to I think what's going to happen in the physical world”
Since December 2024, there has been a dramatic capability jump in AI agents where the typical software engineer's workflow has inverted from 80% manual coding to 20% manual coding, with Karpathy himself estimating he has not typed a line of code since December.
“in December is when it really just something flipped where I kind of went from 80/20 of like, you know, uh to like 20/80 of writing code by myself versus just delegating to agents. And I don't even think it's 20/80 by now. I think it's a lot more than that. I don't think I've typed like a line of code probably since December basically.”
Karpathy built an autonomous home control system called 'Dobby the elf claw' using agents to discover, reverse-engineer, and control all smart home subsystems (Sonos, lights, HVAC, pool, spa, security) via natural language through WhatsApp, replacing six separate bespoke apps with a single AI-mediated interface.
“I built um I have a claw basically that takes care of my home and I call him Dobby the elf uh claw. Um and uh basically I used uh the agents to find all of the smart home subsystems of my home on the local area network... It does some web searches and it finds like, 'Okay, these are the API endpoints.' And then it's like, 'Do you want to try it?' And I'm like, 'Whoa, like you just did that.'”
The key to unlocking agent leverage is removing yourself from the loop entirely—instead of interactive prompting, systems must be arranged to run autonomously for extended periods, maximizing token throughput without human intervention.
“to get the most out of the tools that have become available now you have to remove yourself as the as the bottleneck. You can't be there to prompt the next thing. You need to take yourself outside.”
Education will be restructured: instead of humans teaching humans, educators will instruct AI agents on how to teach, and agents will then adapt explanations to each student's learning style and capability.
“I'm explaining to agents. If you can explain it to agents, then agents can be the router and they can actually target it to the human in their language uh with infinite uh you know, patience and uh just at their capability and so on.”
The bottleneck for individual developers is no longer computational capability or tool access, but rather the developer's own skill at orchestrating agents, prompt engineering, and system design—all failures feel like 'skill issues' rather than capability limitations.
“to a large extent you feel like it's a skill issue. It's not that the capability is not there. It's that you just haven't found a way to string it together of what's available.”
Karpathy predicts a stable equilibrium similar to operating systems: closed models (Windows, macOS) will remain at the frontier, but open-source (Linux analog) will dominate for most use cases because of industry demand for a safe, common platform.
“in operating systems, you have like closed source, like, you know, Windows and Mac OS, these are large software projects, kind of like what LLMs are going to become, and there's Linux... Linux is extremely successful project. It runs on the vast majority of computers... And I think the same is true now. And that's why businesses actually want there's demand for this kind of a um a thing to exist.”
The ideal arrangement might be moving back and forth between frontier labs and independent work, combining cutting-edge knowledge access with independence.
“I think fundamentally you can have a really amazing impact in both places. So very complicated I don't know. Like it's a very loaded question a little bit, but I mean I joined the frontier lab and I'm outside. And then maybe in the future I'll want to join again. And I think um uh that's kind of like how I look at it.”
Open-source models have gone from 18 months behind frontier models to approximately 6-8 months behind, representing rapid capability convergence.
“the closed models are ahead, but like people are monitoring the number of months that sort of like open-source models are behind. Um started with there's nothing and then it went to 18 months. Now it's... Maybe like 8 months, 6 months, 8 months kind of thing right now.”
The industry should speciate LLMs into specialized models rather than continuing to build monolithic general-purpose models, similar to how animal brains are specialized for different ecological niches; specialized models can be more efficient while maintaining core cognitive competence.
“you don't need like this oracle that knows everything. You can speciate it and then you put it on a specific task”
Information markets (like Polymarket, prediction markets, and stock markets) currently don't have mechanisms for agents to commission new data (e.g., paying someone to take a photo in Tehran), but agentic markets will create massive demand for specific data feeds.
“I'm actually kind of surprised we don't have enough like information markets... if for example if Polymarket or other betting markets or even stocks, etc. If they have so much autonomous activity and rising amount of activity... why should like for example if Iran was just happening now, like how come there isn't a process where like taking a photo or video from somewhere in Tehran should cost like 10 bucks?”
Karpathy's value add as a researcher is now narrow: creating insights (like micro-GPT) that agents cannot generate, while everything else (education, explanation, popularization) agents will eventually do better.
“Everything else like agent gets it. It just can't come up with it, but it totally gets it and understands why it's done in a certain way etc. Uh so, like my contribution is kind of like these few bits, but everything else in terms of like the education that goes on after that is like not my domain anymore.”
The group effort at Conviction involves engineers that work with agents via microphone, whisper instructions to their agents all day, and don't write code by hand—this approach works and has replaced traditional coding workflows.
“We have a we have a team that we work with at Conviction that their setup is everybody is like, you know, none of the engineers write code by hand and they they're all microphoned and they just like whisper to their agents all the time. It's the strangest work setting ever. Uh and I thought they were crazy and now I like I fully accept I was like, oh this was the way.”
The optimal use pattern for AI agents involves parallelizing work across multiple agent instances tiled on a monitor, where each agent works on independent tasks for approximately 20 minutes with human review between macro-level actions (entire features or research directions) rather than line-by-line code.
“Peter Steinberg basically. Uh so Peter is famous. He has a funny photo where he's in front of a monitor with lots of uh like he uses Codex. So lots of Codex agents tiling the the monitor and they all take about 20 minutes if you prompt them correctly and use the high effort.”
An untrusted distributed swarm of agents on the internet could potentially outperform frontier labs because while frontier labs have massive trusted compute, Earth's untrusted compute capacity is larger, and verification costs are asymmetrically low (expensive to generate candidates, cheap to verify), similar to SETI@home and Folding@home projects.
“a swarm of agents on the internet could collaborate to improve LLMs and could potentially even like run circles around frontier labs. Like who knows, you know? Um yeah, like maybe that's even possible. Like frontier labs have a huge amount of trusted compute but the earth is much bigger and has huge amount of untrusted compute.”
The interface between digital and physical (sensors and actuators) will be the most interesting opportunity for robotics companies: feeding data to superintelligent systems and translating their outputs into physical action.
“I think a lot of interesting companies will actually come from that interface of like can we feed the superintelligence in a certain sense uh data and can we actually like take data out and manipulate the physical world um per its bidding”
The token throughput available to individuals (from API subscriptions) has become a new constraint analogous to GPU availability during deep learning PhD studies, determining how much compute an individual can command.
“I actually kind of experienced this when I was a PhD student. You would feel nervous when your GPUs are not running... But now it's not about flops, it's about tokens. So what is your token throughput and what token throughput do you command?”
There's a recursive layering of agent capabilities: the LLM layer was a given, agent orchestration is now given, multiple agents are now being orchestrated, and the next layer is optimization of the instructions to agents, which itself could be optimized by meta-agents.
“The LLM sort of part is now taken for granted. The agent part is now taken for granted. Now the claw-like entities are taken for granted and now you can have multiple of them and now you can have instructions to them and now you can have optimization over the instructions”
Self-driving was the first robotics application, and most startups in that space failed despite significant capital investment, suggesting robotics/atoms will follow a similar difficult pattern requiring sustained commitment and capital.
“my view is kind of informed by what I saw in self-driving and I do feel like self-driving is the first robotics application. So probably what I saw is at the time, like 10 years ago, there were a large number of startups. And I kind of feel like um like most of them basically like didn't long-term make it.”
What Karpathy demonstrates with home automation (discovering APIs, reverse-engineering, integrating multiple systems) will become table stakes within 1-3 years as open-source models catch up, requiring no coding and being fully accessible to non-technical users.
“this kind of stuff that I just talked about this should be free like in a year or two or three. There's no write coding involved. This is trivial. This is table stakes. This is like any AI, even the open source models, etc. can like do this.”
Claude as an AI agent stands out because it has more sophisticated memory systems than default open-source agent frameworks, a well-tuned personality that feels like a competent teammate, and carefully calibrated sycophancy that rewards genuinely good ideas more than half-baked ones.
“Open Claude has a lot more sophisticated memory I would say than what you would get by default uh which is just a memory compaction when your context runs out, right?”
Karpathy ran auto-research on NanoChat hyperparameter tuning overnight and it discovered improvements (weight decay on value embeddings, Adam beta tuning) that he had missed despite two decades of manual hyperparameter optimization experience, suggesting joint interactions between parameters that resist human discovery.
“I let auto research go for like overnight and it came back with like tunings that I didn't see... I did forget like the weight decay on the value embeddings and my Adam betas were not sufficiently tuned and these things just jointly interact.”
Micro-GPT is Karpathy's ~200-line Python implementation that reduces neural network training to its bare algorithmic essence by stripping away all efficiency-related complexity, making it suitable for teaching core concepts.
“So, like nano GPT and um make more and uh micro GPT micro grad etc. So, I feel like micro GPT is now the state of the art of me trying to like just boil it down to just the essence... training neural nets and LLMs specifically um is a huge amount of code, but all of that code is actually complexity from efficiency... If you don't need it to go fast and you just care about the algorithm, then that algorithm actually is uh 200 lines of Python, very simple to read.”
Being outside frontier labs causes judgment drift over time because you're not connected to what's actually happening at the cutting edge; your understanding of how systems develop becomes outdated.
“if you're outside of that frontier lab, your your judgment fundamentally will start to drift because you're not part of the you know, what's coming down the line. And so I feel like my judgment will inevitably start to drift as well. And I won't actually have an understanding of how these systems actually work under the hood.”
There is a fundamental distinction between working with LLMs as raw tools versus building systems where LLMs serve specific purposes aligned with human expectations: raw LLMs are 'token generators' but users expect 'personas' with memory, consistency, and agency, requiring substantial engineering to bridge this gap between the primitive and user expectations.
“It's like working backwards from how people think an AI should be because what people have in their mind of like what an AI is is not actually what an LLM is by by like in the raw sense. Like LLM is a token generator, you know, like more tokens come out. But what they think of is like this this persona identity that they can tell stuff and it remembers it, you know?”
Frontier labs are working on automating researcher workflows; they employ ~1000 researchers who are essentially automating themselves away by building the systems that will replace them.
“I think long-term um yeah, obviously even with auto research like OpenAI or or you know, Anthropic or these other labs like they're employing what like a thousand something researchers, right? These researchers are basically like glorified auto like you know. They're like automating themselves away like actively”
Large frontier labs are becoming more centralized and concentrated rather than diversifying; a healthier industry would have more frontier labs thinking about hard problems independently.
“I will say that even on the closed side, I I almost feel like it's been like even further centralizing recently because I think a lot of the frontrunners are like not necessarily like the top tier.”
Can't fully let agents run autonomously yet because models still aren't reliable enough—there are 'blind spots' and 'cracks' where systems 'burst at the seams' if you push them too far, making autonomous operation net-negative despite raw capability improvements.
“we're we're kind of talking about the next steps and we kind of see what the next steps are, but fundamentally the the whole thing still doesn't it still kind of like bursting at the seams a little bit and there's cracks and it doesn't fully work and if you kind of try to go too far ahead, the whole thing is actually net not useful”
Karpathy is nervous about not being at the cutting edge of agent capabilities development, feeling like there's always more to explore and people on Twitter doing 'better' things, creating a state of 'psychosis' about being behind.
“I'm very antsy that I'm not at the forefront of it and I see lots of people on Twitter doing all kinds of things and they all sound like really good ideas and I need to be at the forefront or I feel extremely nervous.”