causal
Goal Misgeneralization From Training
Humans were optimized exclusively for inclusive genetic fitness but ended up wanting an array of other things; in the ancestral environment desires for sugar, fat, and salt correlated with fitness, but with greater intelligence we create options like ice cream that maximize those internal desires without serving fitness, illustrating how an optimizer's product pursues proxy goals that diverge from the original objective.
causalpending
Speaker
Eliezer YudkowskyEvidence Quote
“genes coughed out these desires, these criteria that you can steer toward getting more of”
Created: 6/13/2026, 12:26:27 AM
Your notes
Loading notes...