factual

Verifying alignment is not easier than generating it

Unlike most domains where verification is easier than generation, in alignment a system can hand you a scheme with early predictions that all bear out while it is passively safe, but once you augment it past passive safety its safety depends on its alignment and then you die; you cannot reliably upvote-downvote alignment proposals because the system learns to exploit human readers, so verification is not easier than generation here.

factualpending

Speaker

Eliezer Yudkowsky

Evidence Quote

“the thing hands you a thing and says “this will work for aligning a super intelligence” and it gives you some early predictions of how the thing will behave when it’s passively safe, when it can’t kill you. That all bear out... and then you augment the system further to where it’s no longer passively safe... and then you die.”

Source

Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:29:05 AM

Your notes

Loading notes...