315 | Branden Fitelson on the Logic and Use of Probability
What this covers
Branden Fitelson and Sean Carroll discuss how probability should be understood and applied to reasoning about evidence. The conversation centers on Fitelson's argument that argument strength has two irreducible dimensions—how probable a hypothesis is given evidence, and how relevant that evidence is—and that failing to keep these distinct explains longstanding confusions in logic and psychology. The core claim is that many classic reasoning errors arise from conflating these two measures, a mistake that appears in everything from probability textbooks to how ordinary people judge evidence from medical tests.
The conversation ranges across Hume's Problem of Induction, the philosophy of science, and formal confirmation theory. Fitelson argues that probability is not a matter of limiting frequencies but rather whatever a scientific theory says it is—like mass in physics. He examines how the Bayes factor dissolves famous paradoxes, including Hempel's raven problem and the conjunction fallacy illustrated by the Linda problem. He also challenges Carnap's goal of finding a single universal probability function, arguing instead that the right function must be tailored case by case through empirical modeling. Along the way, Fitelson and Hawthorne connect their account to cognitive psychology, showing how people's errors on tasks like the base rate fallacy and the Wason selection task reflect not irrationality but confusion between two genuinely distinct dimensions. The discussion includes disagreement about how Popper's critique of induction should be understood, and why particle physics generates more confirmationally powerful evidence than other sciences.
Fitelson argues that the strength of evidence has two distinct dimensions—how probable a hypothesis is given evidence (posterior probability) and how relevant the evidence is (Bayes factor/confirmation)—and that conflating them explains both classic reasoning fallacies and longstanding mistakes in inductive logic.
- Probability is whatever a scientific theory says it is (like mass), not a limiting frequency; frequencies are how we know probabilities, not what they are.
- Argument strength is an ordered pair (posterior probability, Bayes factor), not a single totally-ordered number, so 'mixed cases' like the base rate and conjunction fallacies confuse people.
- Carnap's dream of one a priori probability function for all arguments is wrong; instead, for each argument there exists a suitable probability function found through case-by-case scientific modeling.
Likelihoods are objective and invariant, determinable independently of individual beliefs.
- While individuals may legitimately differ in their prior probabilities, they are not entitled to their own likelihoods, because the likelihood information (error rates / Bayes factors) is the objective, invariant component that can be determined in the laboratory independent of any individual's beliefs.
“everyone's entitled to their own priors, no one's entitled to their own likelihoods.”
Prior probabilities and likelihoods jointly determine posterior probability; neither alone suffices.
- Even a highly reliable test with a large Bayes factor will not yield a high posterior probability if the prior is extremely low—a biologically male person getting a positive pregnancy test should not conclude he is probably pregnant because his prior probability is essentially zero, demonstrating that the multiplier alone cannot determine the posterior.
“if I happen to get a positive, I would not conclude that my probability of being pregnant is high 'cause my prior is so low.”
Philosophical maturity involves accepting conclusions as best available without rock-hard foundational certainty.
- Standardized-test questions that give a number sequence and ask for the next one have no unique correct answer, because one can always construct a formula that produces any desired next value after a given finite sequence.
“I can make any number I want. I can come with a formula that would give you any number I want after the ones that you already showed me.”
- A common phase in an individual's philosophical maturation is realizing that not everything can be established on rock-hard foundations agreed to with total certainty, and that sometimes one must accept that a conclusion is simply the best we can do with the available information.
“it's a very common phase in an individual's philosophical maturation to realize that not everything can be established on rock hard foundations”
Science uses non-deductive inference and never achieves mathematical certainty, only provisional best guesses.
- Science never proves things in the sense familiar from mathematics or logic, because it uses non-deductive inference: it looks for patterns in the world while always allowing that tomorrow's experiment or a better theory could overturn the current best guess, in contrast to deductive reasoning that derives conclusions with certainty from premises.
“science never proves things... comes from a comparison to real proof in mathematics, or for that matter, in logic.”