causal

Most negative path: leaked weights after recognizing danger

The most likely bad scenario is that developers proceed, recognize they have a serious problem only after producing a powerful AI they cannot control, and by then the algorithms or model weights have leaked or been stolen, so stopping gives a more careless adversary a decisive advantage.

causalpending

Speaker

Paul Christiano

Evidence Quote

“probably the single most likely negative scenario is that developers proceed and recognize that they have a serious problem. But at the time they recognize it, both the algorithms to develop that AI and quite possibly the weights of the actual model have leaked or been stolen.”

Source

Dwarkesh Patel Podcast - Paul Christiano – We could be 12 months away from AIs that can overthrow the government— Dwarkesh Patel Podcast
Created: 11/17/2026, 6:22:08 AM

Your notes

Loading notes...