causal

Controlling superintelligence is like a perpetual safety machine

Controlling AGI or superintelligence is impossible in the same way a perpetual motion machine is impossible, because systems keep improving, self-modifying, and interacting with malevolent actors, and unlike cybersecurity where you recover from a hack, with existential risk you only get one chance and must create the most complex software ever with zero bugs that stays bug-free for a century or more.

causalpending

Speaker

Roman Yampolskiy

Evidence Quote

“you are really asking me what are the chances that we’ll create the most complex software ever on the first try with zero bugs and it’ll continue to have zero bugs for a hundred years or more.”

Source

Roman Yampolskiy— Lex Fridman Podcast
Created: 6/13/2026, 3:37:06 AM

Your notes

Loading notes...