preferences
In Russell's usage, a person's rankings over all possible ways the future could unfold—what they want the world to be like—not merely tastes between small items.
As defined in 94 | Stuart Russell on Making Artificial Intelligence Compatible with Humans
Where it comes up · 1
Most important claims · 3 of 3Search all claims for “preferences” →
- The proposed solution has two parts: design AI systems whose only objective is whatever humans want the future to be like (their preferences over possible futures), and ensure the system knows it does not know what those preferences are; this uncertainty makes the system willing to be corrected, because human objections convey additional preference information that it is rational to incorporate.94 | Stuart Russell on Making Artificial Intelligence Compatible with Humans
- The hardest conceptual part of the agenda is the gap between the idealized model of a human with stable, consistent preferences and the reality that human preferences change over time; if what I want tomorrow to be differs from what I'll want when I get there, it is unclear which self the AI should satisfy, and there is a danger the AI satisfies preferences by changing them to make them easier to satisfy.94 | Stuart Russell on Making Artificial Intelligence Compatible with Humans
- If AI can do everything humans can do essentially for free, a key danger is idle enfeeblement where humans become couch potatoes who no longer learn or strive; but if the machines genuinely understand our preferences, they will recognize we do not want that future and will refuse to do everything for us—telling humans they must keep learning to do things for themselves.94 | Stuart Russell on Making Artificial Intelligence Compatible with Humans