causal
Most negative path: leaked weights after recognizing danger
The most likely bad scenario is that developers proceed, recognize they have a serious problem only after producing a powerful AI they cannot control, and by then the algorithms or model weights have leaked or been stolen, so stopping gives a more careless adversary a decisive advantage.
causalpending
Speaker
Paul ChristianoEvidence Quote
“probably the single most likely negative scenario is that developers proceed and recognize that they have a serious problem. But at the time they recognize it, both the algorithms to develop that AI and quite possibly the weights of the actual model have leaked or been stolen.”
Source
Dwarkesh Patel Podcast - Paul Christiano – We could be 12 months away from AIs that can overthrow the government— Dwarkesh Patel PodcastCreated: 11/17/2026, 6:22:08 AM
Your notes
Loading notes...