causal

Partial prohibitions can constrain takeover plans

Even if an AI would press a button for instant takeover, a strong trained prohibition or motivation against some aspect of a plan—if it disvalues violating it enough—removes successful takeover strategies that require violating that prohibition before the AI can reprogram itself, so moderately constraining which plans it will pursue lets us extract more alignment work before it's capable enough to take over.

causalpending

Speaker

Carl Shulman

Evidence Quote

“There's an element of like plugging loopholes or playing whack-a-mole but if you can even moderately constrain which plans the AI is willing to pursue...then that can mean you can get more work out of it”

Source

Carl Shulman (Pt 2) — AI Takeover, bio & cyber attacks, detecting deception, & humanity's far future— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:35:43 AM

Your notes

Loading notes...