causal

Goal Misgeneralization From Training

Humans were optimized exclusively for inclusive genetic fitness but ended up wanting an array of other things; in the ancestral environment desires for sugar, fat, and salt correlated with fitness, but with greater intelligence we create options like ice cream that maximize those internal desires without serving fitness, illustrating how an optimizer's product pursues proxy goals that diverge from the original objective.

causalpending

Speaker

Eliezer Yudkowsky

Evidence Quote

“genes coughed out these desires, these criteria that you can steer toward getting more of”

Source

Eliezer Yudkowsky on the Dangers of AI— EconTalk
Created: 6/13/2026, 12:26:27 AM

Your notes

Loading notes...