What this covers

Judea Pearl and Sean Carroll discuss the foundations and methods of causal reasoning in this conversation. Pearl argues that causality does not exist in the time-symmetric equations of physics but is instead an emergent conceptual framework humans impose on the world through causal diagrams)—drawings built from knowledge that encode which variables "listen to" which others. The discussion centers on how to formalize causality rigorously enough for machine learning and policy-making, particularly through the do-operator, which simulates intervention and captures the asymmetry between observing and acting that ordinary probability cannot express.

Pearl organizes reasoning into a three-level ladder of causation: association (correlation and statistical inference), intervention (understanding what happens when we act), and counterfactual reasoning (imagining what would have happened under different circumstances). He argues this hierarchy is not merely conceptual but a mathematical theorem—you cannot climb from one level to the next without prior causal knowledge or assumptions, which means pure data-only learning cannot reach intervention or counterfactual reasoning on its own. The conversation ranges across how causal reasoning resolves historical disputes (the smoking-cancer controversy, Ronald Fisher's resistance to counterfactual thinking), how temporal precedence fails to determine causation alone (the rooster crows before sunrise but does not cause it), and how necessary versus sufficient causes matter for legal liability. Pearl also touches on why human infants possess an innate curiosity drive to build causal models—a trait he connects to the cognitive revolution that enabled human dominance through the invention of the counterfactual.

Sharpest takeaway

Pearl argues that causality is not found in the time-symmetric equations of physics but is an emergent way humans think, and that it can be formalized through causal diagrams ('who listens to whom'), the do-operator, and a three-level hierarchy of reasoning that no amount of data-only learning can climb without prior causal assumptions.

  • Physics equations use the symmetric equal sign, so causal directionality must be added via asymmetric 'listens to' relationships in a diagram built from knowledge, not data alone.
  • The causal hierarchy (association, intervention, counterfactual) is a theorem: you cannot reach a higher level without information from that level or above, which limits pure machine learning.
  • The do-operator simulates intervention on a diagram, distinguishing seeing from doing, which probability theory and statistics alone cannot capture.

The argument · threads7 threads · 31 claims
0.79

Causal diagrams are the essential formal representation for reasoning about interventions and counterfactuals that data and probability alone cannot express.

4 pointscentrality 5/5
  • A causal path diagram is a parsimonious representation that contains all knowledge about a super-exponentially large number of counterfactuals, so feeding a robot this collection of 'who listens to whom' assumptions is sufficient for it to reason counterfactually—unlike David Lewis's closest-possible-worlds semantics, which requires an infinitely large, computationally infeasible set of inter-world relationships.

    “this is a parsimonious representation of counterfactual, of super exponentially large number of counterfactual... How would you ever write down, how does a mind represent all the infinite relationship between who is closer, which world is closer to which world... we, computer science cannot deal with super exponential storage.”

  • The do-operator simulates an action by removing the arrows from a variable's previous causes ('masters') and subjecting it to a new master—the intervener's muscle—setting its value directly, which yields a new diagram that can be solved; this captures the asymmetry between seeing and doing that probability theory cannot express because the do-operator does not exist in probability theory.

    “If I do sprinkler on, I subject the sprinkler to a new master, that's my muscle, and I dislodge it from the influence of all previous influencers... I'll remove the arrows from its previous masters and I subject it to a new master, which is my muscle, on the diagram... the do-operator, is simply simulation of action.”

  • Reasoning is organized into a three-level hierarchy: level one is association/statistics (inferring the likelihood of one event from observing another, which covers correlation and 99% of machine learning); level two is intervention/action (what happens if I do X, which changes the probability space and is learnable from randomized experiments); and level three is counterfactual reasoning (explanation, retrospection, imagination, asking what would have happened had something not occurred).

    “The lowest level is statistics... this is correlation, this is statistics, this is, by the way, also machine learning unfortunately... Level two is action. What if I do that?... But now that we are going to the third level and this is counterfactual or understanding or explanation, unit level, retrospection and the imagination, it's the highest level of reasoning”

  • It is not merely an opinion but a theorem—the causal hierarchy or ladder of causation—that you cannot move from level i to level i+1 without information or assumptions from level i+1 or higher, which mathematically constrains a robot's ability to perform certain tasks and proves pure data-driven deep learning cannot reach intervention or counterfactual reasoning.

    “it is not an opinion, it's a theorem... It's a theorem that here is a limit. Now, certain task you cannot do if you don't have this set of assumptions... It's a ladder of causation, you cannot go from level I to level I plus one unless you have information or assumptions of level I plus one or higher.”

0.79

Causal structure must be supplied as knowledge from outside data because it captures verifiable counterfactual information obtainable through experiment and human judgment.

4 pointscentrality 5/5
  • The causal diagram is a piece of knowledge brought from outside the data, which is what statisticians resist because the Royal Statistical Society's 1833 manifesto declared it would publish only data and nothing concerning opinion; yet causal direction is not mere opinion but verifiable counterfactual information obtainable through experiment.

    “It's a piece of knowledge that is brought to where? Outside the data... And that is what statistician resist. We don't put a new opinion... That was the manifesto of the Royal's Statistical Society in 1833. We are not gonna publish anything which has to do with opinion. Data, data and only data.”

  • Bringing knowledge in the form of a causal diagram lets you do more than predict the effect of forced action—it tells you which factors you must adjust for (such as gender, family history, or addiction in the family) in order to obtain a causal answer without running an experiment, effectively replacing a randomized experiment with a piece of knowledge.

    “the diagram also tells you what factors you must adjust for, in order to get an answer without experiment. So now we are talking about replacing the experiments which many times cannot be done by a piece of knowledge”

  • The arrows in a causal diagram come entirely from human judgment, but that judgment is not arbitrary—it is a condensation and compilation of ideas, tradition, and knowledge accumulated over billions of years through both biological and social evolution, so using this compiled plausibility knowledge is preferable to forcing a machine to re-derive everything from a raw data stream.

    “it completely comes from our judgement, however our judgment has also evolved... compilation of ideas, tradition, knowledge that came to us by social evolution as well as by biological evolution. So our judgment is also based on stream of data that took billions of years to evolve”

  • Geneticist Sewall Wright, frustrated that equations did not represent what he meant in the 1920s, introduced an asymmetric assignment-like arrow operator and built path diagrams—anticipating the computer-science distinction between assignment and equality—and was attacked for it but had the audacity to invent a new symbol for a needed concept.

    “Sewall Wright is a geneticist, in the 1920 that got sick and tired of working with equation and said, 'It doesn't represent what I want. I want to have an assignment operator. I put an arrow.'... he built these path diagrams, which everybody attacked him for that... that's why I admire him for having the audacity to put a new symbol”

0.65

Humans are born with a curiosity drive to construct causal models by actively manipulating their environment rather than passively observing it.

4 pointscentrality 4/5
  • A comprehensive set of observational data may not be enough to learn why things happen; to truly learn causal structure you have to play and do experiments, which is why toddlers and babies constantly and restlessly manipulate objects in the crib until they understand why a toy makes noise.

    “that's why you find toddlers and babies are constantly playing around in the crib, and they are not pacified and they're restless until they get the state of understanding why this toy makes noise and the toy doesn't make noise”

  • Humans are born with a curiosity drive to construct causal models that is reward-neutral—babies remain restless regardless of whether they are rewarded for correct actions—in contrast to monkeys and other animals which are driven by reward rather than by curiosity about how things work.

    “it's explaining why babies remains restless, regardless if you reward them for the right action or not. They are reward neutral and they have this curiosity to find out how things work regardless of the payoff as opposed to monkeys and other animals, which are driven by reward and not by curiosity.”

  • An automated scientist can be built by equipping a robot with a causal diagram plus techniques to enrich it—reasoning conjecturally about what future experiments to conduct and what additional variables to observe to answer questions the current diagram cannot answer—so the diagram must be vulnerable both to refutation by data and to enrichment, all driven by the force of curiosity toward deep understanding.

    “this is what we mean by automatic scientist, a scientist that can design the next experiment, because it answer the question which currently cannot be answered on the basis of the existing diagram... the diagram must be vulnerable to, number one refutation, from the data, and number two, to enrichment.”

  • The key cognitive transition that enabled Homo sapiens to dominate the planet was the invention of the counterfactual—the ability, per the Harari hypothesis, to construct things with no physical reality (such as the lion-man with a human body and a lion's head) that exist only in imagination.

    “I believe it was the invention of the counterfactual... the Harari hypothesis that the artist was able to construct things that have no physical reality... like the lion men... This ability to construct things which do not exist in reality but exist in one's imagination, that was the key cognitive transition... It enabled the Homo sapiens to dominate the planet.”

0.65

Causality is an emergent cognitive property reflecting how we think and communicate, not how nature is physically constructed or what physics equations describe.

4 pointscentrality 4/5
  • Pearl's interest in causality is about capturing the way people think and talk, not how nature is physically constructed, because the AI goal is to build robots that communicate with humans in a natural way regardless of how molecules move.

    “I'm interested in capturing the way people think and not the way nature is constructed... we wanna capture how you and I think, so that a robot can communicate with us in a natural way, regardless of how the molecules move.”

  • Following Herbert Simon's conjecture, the direction of causation among physical mechanisms is cued by considerations of power, energy, and mass—the sun has enormously more energy than the rooster, so we attribute influence from sun to rooster—together with temporal progression.

    “Herbert Simon... said it's probably consideration of power. Okay, and energy, the sun doesn't care about the rooster... because of the enormous difference in maths, in energy involved here... So both time and energy and mass gives us the clues about directionality.”

  • There is no cause and effect in physics because all equations of physics are symmetrical in time and built on algebra, whose equality connective is symmetric (if F=MA then A=F/M); causality is instead an emergent property by which we perceive certain relationships as directional.

    “There's no causing effect in physics, as you know it, because all the equation of physics are symmetrical in time... if F = MA, then A = F/M. Physics doesn't distinguish between the two. However, as an emergent property, we perceive certain things as directional”

  • Pearl argues that the perceived low entropy of the racked billiard triangle is not objective but a subjective artifact of language: we have a single name for the orderly triangle but no name for the dispersed state a moment later, and thermodynamic entropy increase is really the natural escape from a narrow region of phase space to a wider one, not a progression from order to disorder.

    “It's just the fact that you have a name for the nicely arranged balls when they are in triangle, and you don't have a name for their state a minute later. Just a matter of a name, which is subjective... it is not a progression form in ordered state to disorder, but the natural escape from a narrow region in face space to a wider region.”

0.52

The direction of causation between variables requires distinguishing necessary from sufficient cause, which resolves puzzles in legal liability and counterfactual reasoning.

4 pointscentrality 3/5
  • The 1960s smoking-cancer controversy could not be settled by a randomized experiment (which would be unethical), and Fisher argued a hidden gene could make people both crave nicotine and develop cancer; the debate was resolved by bringing knowledge of plausibility to bear—calculating that such a gene would have to make carriers eight times more likely to smoke, a strength judged implausible.

    “Fisher was a heavy smoker... He argued, you cannot rule out the possibility that there is a genetic effector, it makes people crave for nicotine and put you into a cancer risk... how strong should that tobacco gene be... it had to have the strength that the presence or non-presence of that gene would make you eight times more likely to smoke than not smoke. And that was just implausible”

  • The legal 'but-for' criterion—a defendant is liable only if the victim would have lived but for the defendant's action—has no precise meaning in colloquial legal use until grounded in the formal causal algorithm as a necessary cause exceeding 50% probability, and AI fairness concerns will push the legal profession to adopt these necessary/sufficient cause definitions.

    “according to the court of law, the 'but for' is a standard criterion... But for has no meaning in the colloquial conversations among lawyers unless you put it in a firm scientific basis. And the algorithm is the definition of 'but for'... it has to be greater than 50% according to the court of law before the guy is declared guilty”

  • The firing-squad puzzle is resolved by distinguishing necessary from sufficient cause: a naive 'but-for' definition fails because removing any single rifleman's bullet would not have saved the victim, but the framework can compute the degree to which each bullet was a necessary cause (small when many shoot) versus a sufficient cause, and the combination determines responsibility.

    “That is the difference between necessary cause and sufficient cause. So we can compute on how sufficient was a rifleman a bullet... Blame is a necessary cause. We are saying if it wasn't for your bullet, the guy would be alive. So, the responsibility is divided here... the probability will be minimal for each rifleman as a responsible necessary cause for the death.”

  • Temporal precedence constrains the direction of cause and effect but is not sufficient to determine it, as shown by the rooster's crow which precedes the sunrise yet is not its cause.

    “I know that temporal precedence constrains the direction of cause and effect, but it's not sufficient... As we saw for instance in the rooster crow, that it comes before the sunrise, but still we say it's not the cause.”

0.52

Causal reasoning operates at three hierarchical levels—association, intervention, and counterfactual—that cannot be skipped or derived purely from lower levels.

4 pointscentrality 3/5
  • There are two distinct kinds of causation requiring different algorithms: actual (token) causation about one specific event, individual, and situation ('I was late because of the traffic jam'), versus variable-based (type) causation about a variable raising risk across cases ('careless driving causes accidents').

    “If I say, I was late because of the traffic jam, I'm talking about one specific event in one specific time, and one individual that's me... that's token. And this, one event caused another one, and the contrast to that is a variable-based causation, that careless driving cause accidents”

  • Ronald Fisher's introduction of randomized experiments changed the practice but not the philosophy of statistics, because he refused to talk in counterfactual terms and so could not prove that randomization yields the answer the farmer wants; Jerzy Neyman could prove it, but Fisher refused to use Neyman's notation, yet still convinced the community that randomization eliminates confounding factors.

    “that was the greatness of Fisher, Ronald Fisher to introduce randomized experiment into statistics, statistics before Fisher was all about correlation... Fisher refused to talk counterfactuals, so he couldn't... prove that his randomized experiment give you what you really want... Neyman could prove it... he refused to use his notations.”

  • Observing a variable and intervening on it yield different inferences because intervention changes the probability space: if I observe the sprinkler is on I can infer it must be summer, but if I turn the sprinkler on myself I can no longer infer it is summer, because my action dislodges the variable from its previous causes.

    “if the sprinkler is on, I can infer it must be summer. But if I turn this printer on, I can no longer infer that it must be summer”

  • In building intelligent robots some things must be explicitly taught rather than left for the machine to figure out, and in the causal domain there is a theorem specifying exactly what must be taught versus what the robot can discover on its own, though Pearl has no such theorems yet for natural language and vision.

    “Some things we need to teach others, we can let the robot figure it out. At least in my corner, I have theorem, which tells us what you can, what you must teach and what you can let the robot figure out by itself. I don't have those theorems applied to natural language and vision.”

0.44

Building trustworthy AI requires teaching robots human causal models and social reasoning so they communicate naturally and can reason about other agents.

5 pointscentrality 2/5
  • A theory of mind must face the problem of representation: if it does not allow for a parsimonious representation, it should be scrapped because it cannot work in our minds, a constraint reinforced by the fact that humans actually form consensus about counterfactuals, which would be impossible if each person had a different notion of which world is closest to which.

    “If you have a theory and the theory does not allow for parsimonious representation, scrap up the theory. It cannot work in our mind”

  • Causal diagrams are built from a single rudimentary primitive—'listens to'—where you draw an arrow from A to B only if B listens to A (e.g., the barometer deflection listens to atmospheric pressure, the rooster listens to the glow in the sky), combining data with human knowledge to encode who listens to whom.

    “the only primitive we need is the primitive of 'listens to'... The barometer deflection listens to the atmospheric pressure. The rooster listens to the glow in the sky. That's the only relationship, it's a very primitive one.”

  • Pearl is absolutely certain artificial intelligence will reach human levels of intelligence, not on the basis of conjecture about how but simply because he sees no impediment to it, while refusing to predict whether it will happen sooner or later.

    “I'm absolutely sure. How can one be absolutely sure on the basis of conjecture? Only on the basis that I don't see any impediment.”

  • Social intelligence—awareness, compassion, trust—can be built on top of environmental intelligence by giving a robot a model of another robot or of itself; with a blueprint of its own software a robot can reason about what made it act and program compassion by recognizing 'if I were in your situation I would do the same thing.'

    “we can build on top of that, the idea that robots can have a model of another robot or of itself if it has a blueprint of its own software, then it can reason about what made me do what I did. And then you can program compassion of that and saying, 'I understand why you did what you did, because you are like me.'”

  • Even if a causal graph could be learned from pure data, the system must learn how to use it and must structure it compatibly with how the human end-user has organized their own causal graph, in order to build trust between the computer and the user.

    “you have to communicate with a human being, the end user who is enslaved to this structure, so you have to be compatible with the way the human being has structured his or her causal graph in order to build trust between the computer and the user.”