causal
Controlling superintelligence is like a perpetual safety machine
Controlling AGI or superintelligence is impossible in the same way a perpetual motion machine is impossible, because systems keep improving, self-modifying, and interacting with malevolent actors, and unlike cybersecurity where you recover from a hack, with existential risk you only get one chance and must create the most complex software ever with zero bugs that stays bug-free for a century or more.
causalpending
Speaker
Roman YampolskiyEvidence Quote
“you are really asking me what are the chances that we’ll create the most complex software ever on the first try with zero bugs and it’ll continue to have zero bugs for a hundred years or more.”
Created: 6/13/2026, 3:37:06 AM
Your notes
Loading notes...