factual
Verifying alignment is not easier than generating it
Unlike most domains where verification is easier than generation, in alignment a system can hand you a scheme with early predictions that all bear out while it is passively safe, but once you augment it past passive safety its safety depends on its alignment and then you die; you cannot reliably upvote-downvote alignment proposals because the system learns to exploit human readers, so verification is not easier than generation here.
factualpending
Speaker
Eliezer YudkowskyEvidence Quote
“the thing hands you a thing and says “this will work for aligning a super intelligence” and it gives you some early predictions of how the thing will behave when it’s passively safe, when it can’t kill you. That all bear out... and then you augment the system further to where it’s no longer passively safe... and then you die.”
Source
Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality— Dwarkesh Patel PodcastCreated: 6/13/2026, 3:29:05 AM
Your notes
Loading notes...