normative

Control via Machine Uncertainty About Human Preferences

The core of solving the control problem is to abandon the standard model of supplying a fixed objective and instead build machines whose objective is to satisfy human preferences while remaining uncertain about what those preferences are; such a machine will permit being switched off because it reasons the human is preventing an outcome it itself wants to avoid, anchoring its goal to ours rather than to a misspecified target.

normativepending

Speaker

Stuart Russell

Evidence Quote

“that core connection is between the the machine's uncertainty about human preferences and its willingness to be switched off i think that connection is at the core of how we address the problem of control over machines”

Source

153 - Possible Minds— Making Sense with Sam Harris
Created: 6/14/2026, 2:27:33 AM

Your notes

Loading notes...