causal

Enumerating failure modes can inflate probability

The act of enumerating all the specific ways something can go wrong can itself irrationally increase one's estimated probability of failure, because you convert a vague probability cloud into a detailed-seeming list, even though the list is just unpacking the same prior—so detailing AI failure modes need not raise the rational probability of misalignment.

causalpending

Speaker

Dwarkesh Patel

Evidence Quote

“Whereas you had a vague cloud of 10% or something and you're just listing out what the 10% actually constitutes.”

Source

Joe Carlsmith — Preventing an AI takeover— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:37:01 AM

Your notes

Loading notes...