Sheela
Sheela
Home
Add
Questions
Thinkers
Claims
People
Concepts
Jargon
Sources
Health
Predictions
Collections
Domain Landscape
Contribute
Review
Dedup
Topic requests
Settings
Help

← Jargon

Jargonphilosophy· 2 claims· 1 episode

epistemic uncertainty

Uncertainty that cannot be resolved by more thinking alone and requires actual empirical experience to reduce.

As defined in 94 | Stuart Russell on Making Artificial Intelligence Compatible with Humans


Where it comes up · 1
  • 94 | Stuart Russell on Making Artificial Intelligence Compatible with Humans2 claims

Most important claims · 2 of 2Search all claims for “epistemic uncertainty” →
  • Humans have genuine epistemic uncertainty about their own preferences that no amount of thinking can resolve—only actual experience can—exemplified by the durian fruit (which people either love or despise but cannot know without tasting) and by a 16-year-old choosing between being a coal miner or a librarian, so an AI system can never reach perfect knowledge of human preferences in any finite time.94 | Stuart Russell on Making Artificial Intelligence Compatible with Humans
  • Because preference-learning resembles Bayesian inference, the AI approaches certainty only as it approaches correct beliefs about human preferences—provided the prior is broad enough to encompass every possible human preference structure—so a well-designed system that stops asking permission does so because it correctly knows the answer, becoming like a perfect butler (Jeeves) who anticipates needs.94 | Stuart Russell on Making Artificial Intelligence Compatible with Humans