p(doom)
The estimated probability that AI eventually causes human extinction or catastrophe.
As defined in Guillaume Verdon
Other definitions
Shorthand for the estimated probability that AI will cause catastrophic harm to humanity; Cronin argues it is meaningless without a known mechanism.
Lee Cronin
Where it comes up · 10
- AI 2027: month-by-month model of intelligence explosion — Scott Alexander & Daniel Kokotajlo4 claims
- AI 2027: month-by-month model of intelligence explosion — Scott Alexander & Daniel Kokotajlo2 claims
- Sundar Pichai2 claims
- Nick Bostrom: What Happens When AI Evolves Faster Than Humans?1 claim
- 455 - More From Sam Guns Propaganda Ai And Power Unbound1 claim
- 389 - The Politics Of Risk1 claim
- Guillaume Verdon1 claim
- Lee Cronin1 claim
and 2 more
Most important claims · 10 of 15Search all claims for “p(doom)” →
- Although the underlying existential risk from AI is genuinely high, p(doom) is self-modulating: if the risk is actually high, all of humanity aligns to prevent it, and aligned human incentives applied to a focused goal can achieve almost anything, so faith in humanity rising to meet the moment makes the actual doom outcome less likely.Sundar Pichai
- The probability of human self-destruction without AI should be weighed against p(doom) with AI, because AI may make us smarter, kinder, and more efficient and reduce resource constraints that are a common source of military conflict — so AI could be the thing that saves human civilization from its other existential threats.Sundar Pichai
- The most shocking feature of AI leaders' communication is that when asked the probability this technology destroys all human life, they give a very high figure—something like 20%—meaning the people pushing the technology forward with trillions of dollars believe there is roughly a one-in-five chance we are all riding off a cliff.455 - More From Sam Guns Propaganda Ai And Power Unbound
- The probability of AGI doom is zero because, unlike an asteroid impact whose mechanism and prior occurrence we know, we have no mechanism for AGI; assigning a probability to AI doom is like assigning a probability to 'anti-gravity' suddenly being discovered—there is no known mechanism so no meaningful probability can be calculated.Lee Cronin
- If AI proves as disruptive as its makers claim, Silver is genuinely worried; surveys of experts cluster a probability of AI-caused ruin in the 5-10% range, and Silver uses a wider 2-20% range to acknowledge that the smartest people in the space genuinely disagree, making him moderately to seriously worried about creating machines that think substantially better than humans on a rapid timescale.389 - The Politics Of Risk
- The scenario relies on an arms race with China as the primary geopolitical driver that accelerates U.S. deployment of superintelligence-driven capabilities; in absence of such competition, regulatory and safety concerns might slow deployment sufficiently to allow alignment work, but the arms race dynamic makes this much less likely.AI 2027: month-by-month model of intelligence explosion — Scott Alexander & Daniel Kokotajlo
- Daniel's probability of catastrophe (P(doom)) is around 70%, driven by the need for many things to go right simultaneously: not unilaterally slowing down and letting China lead, not racing all-out (which loses control of AIs), and negotiating a power-sharing truce among elites to avoid dictatorship — all of which must happen together.AI 2027: month-by-month model of intelligence explosion — Scott Alexander & Daniel Kokotajlo
- Scott's P(doom) is around 20%, lower than the team's, partly because he isn't convinced we won't get alignment by default and partly because there's a race between AIs delivering an alignment solution (e.g., mechanistic interpretability) and our control over them failing — but his figure excludes oligarchy outcomes, so the surviving 80% still contains many bad worlds.AI 2027: month-by-month model of intelligence explosion — Scott Alexander & Daniel Kokotajlo
- Daniel Kokotajlo's core probability of AI-related doom is 70%, driven by the need to thread a needle between (1) slowing down excessively and letting China win, (2) racing too fast and losing control of AI alignment, and (3) managing concentration of power; all three must be solved for a good outcome.AI 2027: month-by-month model of intelligence explosion — Scott Alexander & Daniel Kokotajlo
- Even if Lex's optimistic view is largely right, assigning it only a 90% chance of being correct means we are betting all of humanity on a 10% chance of catastrophe, which is irrational given the stakes.Roman Yampolskiy