causal

Deference Behaviors Emerge From Uncertainty

Behaviors like asking permission, deferring to human instruction, and allowing oneself to be switched off do not have to be programmed in as special rules; they are logical consequences of an AI being uncertain about the human's true preferences, because a machine that doesn't know whether its plan is wanted will rationally prefer to be corrected or switched off rather than risk doing something the human dislikes.

causalpending

Speaker

Stuart Russell

Evidence Quote

“Those behaviors are just a logical consequence of the way we frame the problem. They are rational for the AI system to do.”

Source

94 | Stuart Russell on Making Artificial Intelligence Compatible with Humans— Sean Carroll's Mindscape
Created: 6/13/2026, 12:03:44 AM

Your notes

Loading notes...