factual

Changing Preferences Pose the Hardest Problem

The hardest conceptual part of the agenda is the gap between the idealized model of a human with stable, consistent preferences and the reality that human preferences change over time; if what I want tomorrow to be differs from what I'll want when I get there, it is unclear which self the AI should satisfy, and there is a danger the AI satisfies preferences by changing them to make them easier to satisfy.

factualpending

Speaker

Stuart Russell

Evidence Quote

“how do you make sure that the AI system doesn't satisfy preferences by changing those preferences to make them easier to satisfy?”

Source

94 | Stuart Russell on Making Artificial Intelligence Compatible with Humans— Sean Carroll's Mindscape
Created: 6/13/2026, 12:03:44 AM

Your notes

Loading notes...