factual

Epistemic Uncertainty About Our Own Preferences

Humans have genuine epistemic uncertainty about their own preferences that no amount of thinking can resolve—only actual experience can—exemplified by the durian fruit (which people either love or despise but cannot know without tasting) and by a 16-year-old choosing between being a coal miner or a librarian, so an AI system can never reach perfect knowledge of human preferences in any finite time.

factualpending

Speaker

Stuart Russell

Evidence Quote

“it doesn't matter how much I think about it, I actually have to go and experience something”

Source

94 | Stuart Russell on Making Artificial Intelligence Compatible with Humans— Sean Carroll's Mindscape
Created: 6/13/2026, 12:03:44 AM

Your notes

Loading notes...