forecast

Misalignment precedes novel destructive tech

Misalignment becomes a catastrophic issue before AI is sophisticated enough to discover destructive technologies not on our radar, because the competence needed to bring down civilization when broadly deployed (with a billion copies acting) is much lower than the competence needed to advise one person how to do so; thus alignment is the sequenced existential bottleneck, with bioweapons a possible exception.

forecastpending

Speaker

Paul Christiano

Evidence Quote

“The level of competence necessary to, if broadly deployed in the world, bring down a civilization is much smaller than the level of competence necessary to advise one person on how to bring down a civilization just because in one case you already have a billion copies of yourself”

Source

Paul Christiano — Preventing an AI takeover— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:36:05 AM

Your notes

Loading notes...