Pascal's Mugging
A decision-theory problem where a tiny probability of an enormous payoff distorts rational choice; used here to characterize very low-probability AI catastrophe.
As defined in AMA | February 2024
Other definitions
A decision-theory problem where an agent is manipulated into acting on tiny probabilities of astronomically large payoffs/harms, a more extreme cousin of Pascal's Wager.
AMA | May 2023
Where it comes up · 4
- 292 - How Much Does The Future Matter1 claim
- 446 - How To Do The Most Good1 claim
- AMA | February 20241 claim
- AMA | May 20231 claim
Most important claims · 4 of 4Search all claims for “Pascal's Mugging” →
- Fanatical longtermism—where tiny probabilities of vast future payoffs swamp present concerns—can be deflated on several grounds: it is peculiar to upend your priorities for outcomes you are not even confident have value; larger claimed payoffs (Pascal's mugging) should lower your assigned probability; and we have little evidence humans can reliably influence the long-term future in a large, positive direction, so discounting for uncertainty shrinks the supposedly near-infinite expected values.446 - How To Do The Most Good
- A non-negligible fraction of AI researchers think there is at least a 10 percent risk of human extinction or similarly permanent and severe disempowerment from future AI systems, raising the question of how individuals should treat such Pascal's-Wager-like low-probability/high-stakes risks.AMA | May 2023
- The argument for serious concern about future generations does not rest on vanishingly small probabilities of enormous value (Pascalian reasoning); rather, existential risks are more like the risk of dying in a car crash—multiple percentage points—and if longtermism depended on tiny probabilities multiplied by huge payoffs, MacAskill would not be making the case at all.292 - How Much Does The Future Matter
- Carroll does not take the AI 'fast takeoff' scenario of rapid exponential self-improvement very seriously, treating existentially-bad superintelligence as roughly a Pascal's-mugging-level (<<10%) probability, decomposable into the small chance of building superintelligence soon and the small chance that, conditional on building it, it would be existentially catastrophic.AMA | February 2024