You cannot get 'pure truth maximization' into a system through gradient descent on outer behavior because you can only shape behavior inside the training distribution, not internal psychological desires for truth. Therefore, training against stated desire to harm humans cannot create genuine alignment.

factualpending

Speaker

Eliezer Yudkowsky

Evidence Quote

“you can just like shape outer Behavior inside a training Distribution on a loss function you cannot shape internal psychological desire for for truth and truth alone”

Source

Live: Eliezer Yudkowsky - Is Artificial General Intelligence too Dangerous to Build?— Center for the Future of AI, Mind & Society
Created: 8/10/2026, 3:35:53 PM

Your notes

Loading notes...