Melanie Mitchell on Artificial Intelligence
What this covers
Melanie Mitchell, computer scientist and author, sits down with EconTalk host Russ Roberts to examine where artificial intelligence actually stands and where the hype has outpaced reality. The conversation centers on a fundamental mismatch: machines excel at narrow, explicitly defined tasks—chess, Go, image recognition—but lack the transferable knowledge and common sense that characterize human intelligence. Mitchell argues that what AI calls "learning" is really statistical pattern-matching within bounded domains, and that the real danger lies not in superintelligent machines taking over, but in deploying systems too unintelligent to handle the messy, contextual world they're placed into.
The discussion ranges across the core tensions in AI research. Mitchell explains the Easy Things Are Hard (Moravec-style paradox))—why solving chess required brute computational force while teaching a machine common sense remains vastly harder. She traces how data versus Knowledge are not interchangeable: machines may possess enormous datasets yet fail at problems humans solve instantly because they lack unconscious world knowledge. The conversation also addresses deep neural networks, their Explainability-Performance Tradeoff, the failures of past approaches like Cyc, bias embedded in training data, and why transfer learning between even slightly different tasks remains elusive. Mitchell critiques predictions from Ray Kurzweil about the singularity and challenges the framing of Value Alignment held by figures like Elon Musk and Nicholas Bostrom, arguing that intelligence is not a scalar quantity to be optimized, and that genuine intelligence emerges through cultural embedding rather than technical design.
Mitchell argues that despite hype, current AI achieves narrow, task-specific performance through statistical pattern-matching rather than genuine intelligence, lacking the transferable world knowledge, common sense, and abstraction that define human cognition.
- Machine 'learning' does not transfer across even slightly different tasks the way human learning does
- So-called intelligence depends on vast unconscious world knowledge machines lack ('data is not knowledge')
- The real near-term danger is AI being too dumb yet widely deployed, not superintelligence
Current AI systems lack the broad world knowledge and common sense that humans possess and apply in novel situations.
- Machines can possess vast data yet still fail at problems humans easily solve, because data is not knowledge—they lack the broad world knowledge needed to predict mundane outcomes, such as knowing whether to stop a car for a paper bag versus a child's toy.
“machines can't solve because data is not knowledge.”
- AI exhibits a paradox in which seemingly hard tasks (chess) yield to brute force and computing power, while seemingly easy tasks like common sense are extremely hard—captured by DARPA's Grand Challenge to build a machine with the common sense of an 18-month-old baby, a goal far more elusive than translating languages or playing Go.
“the Grand Challenge is to create a machine that has the common sense of an 18-month-old baby... this is the perfect example of easy things are hard”
- A trained classifier can recognize new instances within the statistical distribution of its training data (a new photo of a dog), but cannot recognize out-of-distribution cases (a cartoon or painting of a dog) or transfer knowledge to a genuinely new domain—so what AI does is not learning in the human sense, since human learning implies using knowledge in novel situations.
“if it's outside of that distribution, it won't be able to transfer its knowledge to that.”
- A short story about a man whose rare burger arrives burned and who storms out is instantly understood by humans—including the implied answer to 'Did the man eat the hamburger?' (no), the sarcasm of 'it's just great,' and idioms like 'bent out of shape'—but machines cannot infer such unstated meaning because they lack the world knowledge to read between the lines.
“We know the man probably didn't eat the hamburger. Even though it's not said in the story explicitly, we can read between the lines. But how do you get machines to do that?”
- Today's object-recognition AI can identify a person, dog, or balloon in a photo but cannot assemble these into the emotional narrative (a soldier reuniting with her dog) that humans grasp instantly, because it lacks the world knowledge to integrate the pieces into a story.
“it won't be able to put pieces together... It won't be able to put together the kind of story we put together when we're looking at visual data”
The real near-term danger is deploying AI systems too narrow and unintelligent to handle the roles they occupy.
- The real near-term danger of AI is not a superintelligence enslaving us, but the deployment of AI that is too dumb—not general or smart enough to be autonomous—into roles it cannot handle, captured by Pedro Domingos's line that the worry isn't AI being too smart and taking over the world, but its being too dumb and having already taken over the world.
“more that were deploying AI that is not up to the task, that is not general or smart enough to be autonomous.”
- AI systems are not reliable when humans set out to fool them: altering a handful of pixels imperceptible to humans can make an algorithm radically misidentify an image (calling a school bus an ostrich), and such adversarial attacks on facial, object, and language systems are not hard to carry out.
“those are called adversarial attacks, because an adversary can attack an AI system. And people have shown that it's actually not that hard to do.”
- AI's accomplishments—beating top chess and Go players, speech and handwriting recognition, machine translation, medical image analysis—are each narrow tasks requiring a system built largely from scratch; none generalizes, so a system that masters Go cannot even play a variation of Go, let alone checkers.
“each application is somewhat narrow and it's a particular--you have to sort of start from scratch with building a system that will do that application rather than having some more general AI”
- The biggest source of accidents for experimental self-driving cars is being rear-ended, because they stop unexpectedly—slamming brakes for things no human would stop for—since they lack the common sense to know what is safe to ignore in situations different from their explicit training.
“the biggest source of accidents in self-driving cars, the experimental ones that are driving around, are people rear-ending them.”
Intelligence is not a single scalar metric but a complex, poorly understood phenomenon requiring multiple fields to address AI's risks.
- Kurzweil's singularity prediction—that by 2045 we'll have intelligences a billion times smarter than humans—rests on extending Moore's Law exponential growth, but software shows no such exponential trend, hardware exponential trends may no longer hold, and the claim wrongly assumes intelligence is a single scalar metric that can be multiplied.
“he's implying that there's some intelligence metric that can be multiplied by a billion. And, intelligence to me is not just a single thing.”
- The fear of superintelligent AI held by Hawking, Musk, and Bostrom rests on a too-simplified view of intelligence—treating value 'alignment' as a malleable switch to flip on—whereas a truly intelligent system would develop within a society and culture and acquire values through that embedding; a superintelligence that solved climate change by killing all humans would be a misconstrual of the word 'intelligent.'
“as if the alignment with values is just this malleable switch that you can turn on and off... rather than thinking that an intelligent system is actually a very complicated thing that sort of develops in a society, in a culture”
- The most exciting thing about AI is the insight it gives into our own intelligence and the nature of intelligence in general; failures are as illuminating as successes—perhaps more—because, like genetic engineering, AI carries both incredible potential benefits and huge risks that require many fields, not just AI experts and executives, to deliberate.
“I get as excited by the failures as I do by the successes because that I think we can learn something from--maybe even more, learn more from the failures than we do from the successes.”
- Intelligence is a 'suitcase word' and a placeholder for phenomena we don't yet understand—it is not a single thing nor a yes/no property, and because our understanding of the brain and human psychology remains limited, the term awaits scientific advances to replace it with something more precise.
“intelligence is the one of those terms that's a placeholder for things we don't understand yet.”
- Human learning may itself be mechanical—neurons firing and memories forming—differing from machines only in degree of complexity and in being evolved to find certain stimuli (like faces and emotions) highly salient, because sociality made those things important to our survival.
“I think in some sense we are mechanical. ... it's a matter of complexity.”
- Human intelligence evolved for specific purposes, and another intelligence need not share our architecture, desires, or goals; but anything meant to operate usefully in the human world—driving among us or acting as a virtual assistant—will have to possess human-like qualities and understanding to deal with us.
“if we want it to be useful to us it's going to have to have the kind of human understanding that we have.”
Hype about AI has driven unrealistic expectations through commercial marketing and human anthropomorphization of systems.
- Hype has plagued AI since its inception and has grown worse as AI works better, driven partly by the commercial need to market the technology (e.g., IBM's Watson advertising) and partly by humans' tendency to anthropomorphize systems with names, voices, and personalities, giving them more credit for understanding than they deserve.
“I think the hype has gotten almost worse, the better AI works.”
- Watson did not 'know' a lot of things so much as it could rapidly look up information on Wikipedia and large databases plus use some natural-language processing to parse Jeopardy questions; its abilities did not transfer to medicine as IBM marketed, and the Jeopardy-winning Watson effectively no longer exists, replaced by a rebranded platform of standard deep-learning tools.
“I would say it has the ability to look up things very quickly on Wikipedia and other big database sites.”
- Joseph Weizenbaum's 1960s Eliza chatbot, despite using only a few keyword templates to mimic a psychotherapist, led people to confide their deepest secrets because they believed it truly understood and listened to them—demonstrating humans' readiness to project understanding onto trivial programs, which horrified Weizenbaum into anti-AI activism.
“It was the most simple program. It had a few templates. It had some keywords. ... people wanted to talk to it. People wanted to tell their deepest secrets.”
- The Turing Test tries to sidestep the unknowable question of whether a machine has inner experience by judging only conversational indistinguishability, but because humans so readily anthropomorphize, the test turns out to be too easy to pass to serve as a meaningful measure of machine understanding.
“the Turing Test tries to get around that. But it turns out to be a little too easy: because of our human propensity to anthropomorphize, it becomes too easy to pass the Turing Test.”
Deep neural networks face an unsolved trade-off between performance and explainability that prevents meaningful transparency.
- There is an almost inverse relationship between a deep neural network's success and its explainability: deeper networks with more layers and more (millions or billions of) weights tend to perform better by modeling data more successfully, but offer no high-level insight into why they make a given decision, making explainability an unsolved problem.
“There's almost an inverse relationship between the success of a system and its explainability at least with these deep neural networks.”
- A deep neural network is a computer program inspired by the brain's layered visual system; 'neural' refers to simulated neurons and 'deep' refers simply to the number of layers, not to profundity—a shallow network has few layers, a deep one has many.
“'deep' refers to how many layers there are. ... that's all that 'deep' means--is sort of how many layers of simulated neurons there are in the network.”
- What is called 'learning' in a neural network is fundamentally the re-weighting of connections between simulated neurons: when the network makes a mistake on labeled data, it adjusts weights so a given pixel pattern becomes more likely to be classified correctly.
“The connections between the simulated neurons.”
Bias in AI systems runs deeply through biased training data and society itself, making de-biasing extremely difficult.
- Bias in AI runs very deep because our society is biased and so is the training data; for example, facial recognition performs worse on darker skin partly due to biased data, but also because camera electronics themselves are tuned better for lighter skin—making de-biasing extremely hard in a society where bias is so deep and almost invisible.
“facial recognition performs worse on darker skin than lighter skin, and that's partially because of the biased data... but also I found out that it's cameras themselves, the electronics and cameras are tuned better for lighter skin”
- Verifying crowd-sourced labels is an unsolved problem: simple tasks can be checked via majority vote among multiple labelers, but for complex tasks like judging whether one sentence entails or contradicts another, human labelers make frequent errors, making the data noisier the harder the task.
“The more complicated the task, the more tricky the whole Mechanical Turk thing is.”
A small number of companies wield troubling power over data and information despite funding valuable basic research.
- A small number of companies wield troubling power because they possess so much data and so much control over what we see, think, and do; this is tempered by the unusual and valuable fact that these profit-driven firms now fund basic research, but their profit motive makes them untrustworthy to consistently do the right thing.
“There's just a few companies that have so much power--because they have so much data... and they have so much kind of control over what we see, what we think, what we do. I find that really troubling.”
- Amazon's Mechanical Turk—named after an 18th-century chess-playing hoax that hid a human inside—pays people small sums to perform tasks too hard for AI, such as labeling images; ironically, the rise of AI has created a large pool of very low-paying human jobs rather than purely eliminating work.
“this idea that AI is going to put people out of a job is actually a little more complicated because AI now, or the lack of thereof, has created this huge set of very low-paying jobs for people who are on this platform.”
- Hofstadter's terror about AI was not that a malevolent superintelligence would enslave us, but that intelligence itself might turn out to be far less profound than he believed—achievable in computers through 'cheap tricks'—which disturbed him as AI excelled at chess, Jeopardy, self-driving, and speech recognition.
“He was worried that intelligence, AI might be achieved in computers via cheap tricks.”
- AI has been split for at least 40 years over an innateness-versus-learning debate—analogous to nature versus nurture—about how much knowledge and programming to build into a system versus how much it should learn on its own; each approach has problems, and surveys of when human-level AI will arrive yield essentially a uniform distribution, reflecting that the field has no clue.
“There's this big fight that's been going on for at least the last 40 years probably about innateness versus learning. So, it's like the nature versus nurture debate.”
- Approaches like Doug Lenat's Cyc tried to give machines common sense by manually writing down a huge database of facts (e.g., 'you can't be in two places at once'), but this fails because so much of human common sense is unconscious and therefore cannot be written down.
“The problem is that we can't write it down because so much of it is unconscious.”
- Deep neural networks existed conceptually since the 1960s-70s but only began to shine in the last decade because two prerequisites finally converged: huge amounts of labeled data from the World Wide Web and fast parallel computing power.
“people never had enough data to train them, and they never had enough compute power to make that training possible.”
AI systems demand explainability and accountability for consequential decisions, just as human decision-makers must justify their reasoning.
- The argument that AI need not explain itself because humans can't explain their thinking is false: while imperfect, humans genuinely can and do explain reasoning when stakes are high—a judge writes a long explanation citing evidence—so AI making life-altering decisions should likewise be required to account for the evidence it used.
“humans can't explain their thinking either, so why should we make AI explain this thinking?' But I think that's actually a false argument. Because, humans can explain their thinking.”
- The notion that teaching AI researchers ethics will solve AI's dangers is wrong on multiple counts: whose ethics, and more fundamentally, ethics is a complex conceptual matter while computers have no concepts—being ethical may be equivalent to being intelligent—so ethics cannot be learned 'on the side' while a system learns a narrow task, making this a systemic societal issue rather than a training fix.
“ethics is a very complex conceptual thing, and computers have no concepts... understanding ethics and being ethical is maybe equivalent to being intelligent”