What this covers

This episode brings together Sean Carroll and Anil Ananthaswamy, a science writer and author of Why Machines Learn: The Elegant Math Behind Modern AI, to examine the mathematical foundations underlying the recent explosion in AI capability. The conversation moves from the history of neural networks—including early work by Bernard Widrow and Frank Rosen blat—through the central insight that modern deep learning rests on remarkably classical tools: backpropagation, which is simply the chain rule of calculus applied in extremely high-dimensional spaces. Ananthaswamy and Carroll establish that this mathematics has remained fundamentally unchanged since the 1986 Rumelhart-Hinton-Williams paper, yet applying it at scale—across trillion parameters—has proven revolutionary.

The discussion covers the range of techniques that make modern AI work: the Curse of Dimensionality and how High-Dimensional Representation of Data enables classification, the Kernel Trick for achieving non-linear boundaries without explicit computation in higher dimensions, and the mechanics of contemporary language models through Attention Mechanism layers that refine weights across feedforward network architectures. Ananthaswamy articulates a more cautious view of AI's trajectory than current hype suggests: while scaling has driven impressive next word prediction performance, he argues that large language models are sample inefficient and lack mathematical guarantees of accuracy, making it implausible that scaling alone will yield general intelligence. He positions genuine scientific breakthroughs—like Kepler's laws or Albert Einstein's relativity—as fundamentally different from pattern interpolation, suggesting that reaching transformative AI will require conceptual innovations beyond the current framework.

Sharpest takeaway

Ananthaswamy argues that the remarkable progress in modern AI rests on surprisingly classical, simple mathematics (linear algebra and calculus) applied in extremely high-dimensional spaces, and that scaling current large language models alone will not reach general intelligence without further conceptual breakthroughs.

  • Core techniques like backpropagation are just the chain rule of calculus, unchanged since 1986 despite scaling to trillions of parameters
  • LLMs are sample-inefficient and offer no mathematical guarantee of accuracy, so scaling cannot reach reliable general intelligence
  • Genuine conceptual leaps (like Kepler's or Einstein's) appear beyond current neural networks, which interpolate from training data

The argument · threads8 threads · 26 claims
0.78

Scaling large language models alone is insufficient for generalized intelligence.

3 pointscentrality 5/5
  • Scaling up large language models alone will not produce generalized intelligence because there is no mathematical guarantee a language model is accurate—it outputs a probability distribution with inbuilt stochasticity and only asymptotically approaches correctness—and because LLMs are extremely sample-inefficient; reaching transformative AI capable of generalizing beyond training data likely requires two or three further breakthroughs comparable to the attention paper.

    “we have no mathematical guarantee that a language model is 100% accurate. That you cannot guarantee accuracy... there is an inbuilt stochasticity there.”

  • Large language models can plausibly automate routine science—writing analysis code, generating figures, finding references, and drafting a paper from a dataset—but they appear unable to make genuinely new conceptualizations, such as deriving Einstein's relativity from pre-1915 data, which is a fundamentally different kind of scientific creativity.

    “if we had an LLM that had data about physics that happened until 1915 and LLM. And then could it then come up with Einstein's theory of relativity without having anything in the data about relativity? Very unlikely. I wouldn't even say unlikely. I would say impossible at this point.”

  • A deep neural network given the same sparse data Kepler had (only tens of orbital positions) cannot reproduce Kepler's laws, because today's neural networks are extremely sample-inefficient and require far more data than humans, and Kepler also brought substantial prior knowledge of geometry and calculus.

    “There's no way a neural network would do what Kepler did. Kepler had access to literally a few tens of positions of the orbits of Mars and Jupiter.”

0.71

Backpropagation and gradient descent are the mathematical foundation enabling neural network training at any scale.

4 pointscentrality 4/5
  • Bernard Widrow's adaptive digital filter work led to the Widrow-Hoff Least Mean Square algorithm, which is the true precursor to today's backpropagation; it was effectively a very noisy, algebraic version of stochastic gradient descent that avoided calculus so it could be implemented directly in hardware.

    “the algorithm that Widrow came up with to build this linear classifier is actually the true precursor to today's back propagation algorithm”

  • The backpropagation and gradient descent math is identical whether a network has a handful of parameters or close to a trillion (as in GPT-4, O1, O3, Claude); the loss landscape simply becomes a function of that many parameters, and the core algorithm from 1986 still holds despite modern engineering innovations like stochastic gradient descent and momentum.

    “the math is the same regardless of whether it's a trillion parameters or four parameters. That's the amazing part. The back propagation algorithm, what Hinton did in 1986 in their paper... the same stuff holds true today.”

  • The 1986 Rumelhart-Hinton-Williams backpropagation paper showed that multi-layer neural networks can be trained using what turns out to be just the chain rule of calculus, which requires that the entire computational graph from output to input be differentiable—achieved by replacing the non-differentiable step threshold with a smooth sigmoid.

    “it was Hinton, Rumelhart and Williams who in 1986 put out a paper in Nature... that is essentially the back propagation algorithm, which shows you how you can train a multi-layer neural network using what turns out to be just the chain rule of calculus.”

  • A language model's output is a probability distribution over its vocabulary for the next token, from which a token is sampled; during training the target distribution is one for the correct word and zero elsewhere, and backpropagation over hundreds of billions to a trillion parameters gradually nudges the predicted distribution toward the correct word across every sentence scraped from the internet.

    “what the language model is predicting at the very end is a probability distribution over its vocabulary... here's the probability distribution over my vocabulary as to what is the most likely next word.”

0.71

High-dimensional space enables linear separation of non-linearly separable data through projection and kernel methods.

4 pointscentrality 4/5
  • As the number of dimensions (features) grows, distance-based methods like k-nearest-neighbor fail because in high dimensions everything is roughly as far from everything else, so the notion that similar data points are closer than dissimilar ones collapses—this is the curse of dimensionality, predating and orthogonal to neural networks.

    “as you increase the number of dimensions of your data, then there comes a point where the notion of similarity or dissimilarity that this algorithm depends on... starts falling apart because in high dimensions everything is as far away as everything else.”

  • Kernel methods use a kernel function that takes two low-dimensional vectors and returns a scalar equal to the dot product of their high-dimensional mappings, allowing a linear classifier to operate effectively in million- or even infinite-dimensional space (where separation is guaranteed) without ever computing in those dimensions, yielding intricate non-linear boundaries when projected back.

    “you feed in the two lower dimensional vectors and it spits out a number that is actually equal to phi of x dot phi of y. Except you've never gone into the million dimensions.”

  • Principal Component Analysis projects high-dimensional data into fewer dimensions while capturing most of its variance along a few chosen axes; this only works if the data's structure concentrates variance, since if data varied equally across all axes the projection would lose the information needed for classification.

    “do something like PCA, Principal Component Analysis, where you essentially project the data back into lower dimensions and hopefully capture most of the variance in your data along the few lower dimensional axes that you've chosen”

  • Data that is not linearly separable in low dimensions (e.g., red dots surrounded by a green annular ring in 2D) can become linearly separable by projecting it into higher dimensions—such as adding a squared-coordinate axis that lifts the green ring above the red center—after which a hyperplane separates them and, projected back, yields a non-linear boundary.

    “if your red colored dots are centered around the origin and your green colored dots are an annular ring around the red colored dots, there's no straight line that's gonna separate the two clusters in two dimensions. But you can imagine adding a third dimension... and then you can draw a hyperplane between the two.”

0.57

Single-layer perceptrons solve linearly separable problems with theoretical guarantees, but multi-layer networks face non-convex optimization and out-of-distribution failures.

4 pointscentrality 3/5
  • A linear classifier only decides which side of its hyperplane a new input falls on, so when given an out-of-distribution input (e.g., a horse image when trained only on cats and dogs) it has no way to recognize the novelty and will misclassify it as one of the known categories.

    “Now you bring in an image of a horse. The classifier has no idea that. All it does is it says, which side of the hyperplane does this image fall on? And it's gonna call the horse either a cat or a dog.”

  • The simple artificial neuron is a basic approximation of biological neurons: like a cell body accumulating dendritic signals and firing along its axon when the strength and timing cross a threshold, the artificial neuron does a weighted sum plus bias and fires (outputs 1 or -1) only when a threshold is exceeded.

    “you have all these signals coming in through the dendrites of the neuron into the cell body... when it crosses some threshold... it then fires a signal on its own axon... the simple kind of artificial neuron is a very basic approximation of this biological neural mechanism.”

  • Rosenblatt's perceptron, a single-layer artificial neural network, was guaranteed by the perceptron convergence proof to find a separating hyperplane in finite time whenever the data is linearly separable in any-dimensional space, using only basic linear algebra—a landmark guarantee in 1950s computer science.

    “if the data is linearly separable, then the algorithm will find it in finite time. And this was a huge statement to make in computer science terms back in the 1950s, that an algorithm is guaranteed to work”

  • Minsky and Papert's 1969 book Perceptrons proved that single-layer neural networks cannot solve the XOR problem, and by insinuating without proof that multi-layer networks also could not, they effectively killed neural network research and triggered the first AI winter, possibly motivated by their preference for symbolic AI funding.

    “Minsky and Paper had a very elegant proof saying that single layer neural networks will never solve this problem... what they did, which was kind of underhanded, was they also insinuated, without giving any mathematical proof, that even multi-layer neural networks will not solve this problem.”

0.50

Neural networks embed learned patterns but cannot extract or explain symbolic laws underlying their predictions.

3 pointscentrality 3/5
  • Using a trained network (e.g., ChatGPT inference) is purely a feedforward process where an input vector propagates layer by layer to an output, whereas training requires the back-and-forth of computing a loss on the output side and backpropagating it to update each layer's weights toward the input side.

    “for people who are, for instance, using something like ChatGPT today, when we use it, we're just using the feedforward part of it. But when you're training it, you have to keep doing this back and forth.”

  • A trained neural network can embody in its weights something very similar to what Kepler figured out and can predict future planetary positions as a time series, but it cannot spit out a symbolic form of the underlying equation; extracting the symbolic law from the network's weights is an unsolved problem.

    “it has no way of spitting out some symbolic form of Kepler's loss. Right? But the network is embodying in its weights something that is very similar to what Kepler would have figured out. The question is, how do we extract that out of that network?”

  • Because modern neurons include non-linearities (e.g., sigmoids) and networks are enormous, the loss function over a trillion parameters is non-convex and jagged rather than smooth; there is no guarantee it has a global minimum, so training relies on gradient descent finding a satisfactory local minimum among many.

    “it is not a convex function... These lost landscapes are extraordinarily complex. They have probably, we don't even know for sure that they have a global minimum, but they may have lots and lots of very good optimal local minima.”

0.36

Recurrent and associative networks use feedback loops and energy minimization to enable memory retrieval fundamentally different from feedforward architectures.

3 pointscentrality 2/5
  • Hopfield networks are fully connected recurrent neural networks where neuron outputs feed back into the rest of the network, which differs fundamentally from today's feedforward networks in which computation proceeds from input to output without feedback.

    “They're what are called recurrent neural networks because the outputs of a neuron can feed back into the rest of the network... which is not the way networks work in today's neural networks. Today's neural networks are what are just called feed forward networks.”

  • In a transformer, input words are turned into high-dimensional vectors (embeddings) that flow through the network; the attention mechanism repeatedly massages these vectors layer by layer so each word's vector incorporates contextual information from the others, so that by the final layer the last vector carries enough context to predict the next word correctly (e.g., 'homework' after 'the dog ate my').

    “as those four vectors flow through the deep neural network, which is the transformer, it has to contextualize it, has to keep massaging those vectors such that the words start paying attention to each other.”

  • John Hopfield's networks, inspired by the Ising model of ferromagnets and spin glasses, are fully connected recurrent networks where connection strengths are set so a stored memory sits at an energy minimum; perturbing the network drives it to a higher energy state, after which neurons flip like magnetic moments and the system relaxes back down the energy landscape to the stored memory, enabling associative memory retrieval.

    “here the energy of the system would be at a minimum when you stored a memory. And anytime you corrupted the network, it would enter into a higher energy state... the whole system will traverse its way down the energy landscape and then settle back into its minima”

0.29

Scaling works for LLMs because their training process is fully automatable, requiring no human input beyond internet text.

2 pointscentrality 2/5
  • Scaling has worked for LLMs because the training process—scrape internet text, mask the last word, and ask the network to predict it—is essentially hands-off and requires no human input, making it fully automatable and therefore amenable to scaling up.

    “the reason why scaling has worked so far is because this entire process of training a large language model is more or less hands off in terms of human inputs... a process that can be completely automated and hence amenable to scaling up.”

  • Much of the effort in machine learning goes into speeding up processes, because an algorithm that produces amazing results but takes an impractically long time to run (e.g., 10,000 years) is useless; the field therefore requires substantial math-informed engineering.

    “you might find an algorithm that would do amazing things, but if it takes 10,000 years to run, it's not could be very helpful to you in the real world.”

0.07

Transformers process sequential input through embeddings and attention mechanisms that progressively incorporate contextual information across layers.

1 pointcentrality 1/5
  • A black-and-white image can be represented as a single point in a high-dimensional space (a 20x20 image becomes a point in 400-dimensional space, a 1000x1000 image a point in million-dimensional space), with similar images clustering together, enabling a classifier to assign a new image based on which side of a separating hyperplane it falls.

    “if you were to map each pixel along one axis, then in 400 dimensional space, each of these images becomes a bunch of points. And so the image 9, all images of the digit 9 will be in one location of this 400 dimensional space”