normative

Solution: Uncertain Objective of Human Preferences

The proposed solution has two parts: design AI systems whose only objective is whatever humans want the future to be like (their preferences over possible futures), and ensure the system knows it does not know what those preferences are; this uncertainty makes the system willing to be corrected, because human objections convey additional preference information that it is rational to incorporate.

normativepending

Speaker

Stuart Russell

Evidence Quote

“their only objective is whatever it is that humans want the future to be like... it knows that it doesn't know what those preferences are”

Source

94 | Stuart Russell on Making Artificial Intelligence Compatible with Humans— Sean Carroll's Mindscape
Created: 6/13/2026, 12:03:44 AM

Your notes

Loading notes...