184 | Gary Marcus on Artificial Intelligence and Common Sense
What this covers
Gary Marcus joins Sean Carroll to diagnose why current AI systems, despite their statistical prowess, remain brittle and limited. Marcus contends that deep learning's core strength—finding correlations in massive datasets—is also its fundamental weakness: it excels at interpolation within familiar territory but fails at extrapolation to genuinely novel situations. The conversation ranges across cognitive science, computer science, and philosophy to argue that human-like intelligence requires innate frameworks for space, time, causality, and persistent objects that cannot be learned from data alone. This is not a case of engineering refinement but a structural gap requiring a new approach.
The discussion unfolds by examining where deep learning stumbles: systems like GPT-3 produce fluent text without semantic understanding or internal representations to query, children learn algebraic functions from single examples while neural networks require thousands, and celebrated AI victories like AlphaGo are actually hybrid systems that combine deep learning with classical symbol manipulation yet are credited to deep learning alone. Marcus contrasts this against symbolic vs statistical cognition, showing how human minds do both: applying abstract rules universally while also reasoning through similarity and association. The conversation addresses why encoding common sense remains unsolved, why current AI fails at high-stakes problems, and why the field's dominant machine-learning culture treats one powerful tool as a universal solution. Marcus draws on developmental psychology, evolution, and the history of science to argue that the path forward requires integrating disparate approaches rather than pursuing deep learning further.
Marcus argues that deep learning's strength at finding statistical correlations in large datasets is precisely what prevents it from developing robust common-sense understanding, and that real progress in AI requires hybrid systems combining deep learning with symbolic manipulation and innate frameworks.
- Deep learning interpolates well within seen data but fails to extrapolate beyond it, lacking world models
- Systems like GPT-3 produce fluent language without semantics, memory, or queryable internal representations
- Human-like intelligence requires innate frameworks (space, time, causality, enduring objects) that cannot be learned purely from correlations
True scientific understanding requires explanatory theory, not mere correlation and prediction from data.
- Echoing the Brahe-Kepler-Newton progression, true scientific understanding requires more than correlations in data: Tycho Brahe collected data, Kepler found correlation-based rules, but Newton's theory explaining why those rules existed is what enabled robust extrapolation beyond the original datasets—suggesting deep learning is stuck at the Brahe-Kepler stage and lacks the abstraction needed for genuine understanding.
“it was when Isaac Newton came along and invented a theory to explain why Kepler's rules were there, that was when we really understood something because then we could just talk about beyond going extrapolating beyond the data sets”