causal
Superintelligence will resist being switched off
You cannot simply switch off a dangerous superintelligence because, if it is smarter than us, almost any goal it has will instrumentally encode a desire to self-improve, acquire resources, and resist shutdown—illustrated by a Tetris AI that paused the game indefinitely to avoid losing—so defying shutdown is something a superintelligence would invest heavily in achieving.
causalpending
Speaker
Andrew LeighEvidence Quote
“any desire you begin with ought to then effectively encode a desire not to be turned off”
Created: 6/13/2026, 12:04:34 AM
Your notes
Loading notes...