factual

Bayesian Certainty Only Arrives When Correct

Because preference-learning resembles Bayesian inference, the AI approaches certainty only as it approaches correct beliefs about human preferences—provided the prior is broad enough to encompass every possible human preference structure—so a well-designed system that stops asking permission does so because it correctly knows the answer, becoming like a perfect butler (Jeeves) who anticipates needs.

factualpending

Speaker

Stuart Russell

Evidence Quote

“you only approach certainty as you approach having correct beliefs about the human preference structure”

Source

94 | Stuart Russell on Making Artificial Intelligence Compatible with Humans— Sean Carroll's Mindscape
Created: 6/13/2026, 12:03:44 AM

Your notes

Loading notes...