174 | Tai-Danae Bradley on Algebra, Topology, Language, and Entropy
What this covers
Tai-Danae Bradley and Sean Carroll explore how abstract mathematics—category theory, topology, and quantum probability—can illuminate the hidden structure of natural language. Bradley, a research mathematician at Alphabet, argues that words and expressions have algebraic and statistical foundations that reveal meaning and grammar in ways classical methods miss. The conversation moves through a series of concrete examples: how word concatenation forms a monoid, how statistics can stand in for grammar, how the Yoneda Lemma echoes John Firth's principle that meaning comes from context, and how Shannon entropy behaves mathematically like a calculus rule.
The discussion covers the range of structures that emerge when mathematics meets language. Bradley explains how large language models succeed by learning only algebra and statistics—no explicit grammar required—yet the question of what deeper mathematics lies beneath remains open. She introduces the surprising connection between probability and topology: a probability distribution is geometrically a simplex, which reframes entropy as a topological phenomenon. A quantum-style representation using reduced density matrices allows marginal probability with memory, retaining information that classical marginalization discards. Most provocatively, Bradley presents evidence that entropy satisfies a Leibniz-like rule and that the topological principle "boundary of a boundary is zero" recurs across multiple characterizations of entropy—suggesting unifying structure yet to be fully understood.
Bradley argues that natural language has hidden algebraic and statistical structure, and that abstract mathematics (category theory, quantum-style probability, topology) reveals deep, possibly unifying patterns — including that a word's meaning can be captured by the statistics of its context and that Shannon entropy behaves like a calculus derivation.
- Concatenating words is an algebraic (monoid) operation, and statistics over expressions can serve as a proxy for grammar and meaning (Yoneda Lemma / Firth's 'know a word by the company it keeps').
- A quantum-style (density operator) version of probability lets you marginalize while retaining information about the discarded subsystem.
- Entropy is a function on simplices (spaces of probability distributions) and satisfies a Leibniz-like rule, suggesting deep links to algebra and topology.
Professional mathematicians primarily invent concepts and establish connections rather than perform calculations.
- Most of what professional mathematicians do is not calculation but inventing new concepts, combining them to prove theorems, and drawing previously unknown connections between concepts.
“professional mathematicians are engineers of concepts. They invent new kinds of concepts, they put them together to prove theorems, and they draw connections between concepts that you might not have known about before.”