You cannot get 'pure truth maximization' into a system through gradient descent on outer behavior because you can only shape behavior inside the training distribution, not internal psychological desires for truth. Therefore, training against stated desire to harm humans cannot create genuine alignment.
factualpending
Speaker
Eliezer YudkowskyEvidence Quote
“you can just like shape outer Behavior inside a training Distribution on a loss function you cannot shape internal psychological desire for for truth and truth alone”
Source
Live: Eliezer Yudkowsky - Is Artificial General Intelligence too Dangerous to Build?— Center for the Future of AI, Mind & SocietyCreated: 8/10/2026, 3:35:53 PM
Your notes
Loading notes...