causal

Value Alignment And The King Midas Problem

The value alignment problem is ensuring a machine's optimized values match what humans really desire; because machines are optimizers, an incompletely specified objective will be pursued in unexpected, harmful ways—the King Midas / genie pattern—and with superintelligent AI you may not get a second or third wish to correct the mistake.

causalpending

Speaker

Stuart Russell

Evidence Quote

“nowadays we call this the value alignment problem how do we make sure that the the values that the machine is trying to optimize are in fact the values of the human”

Source

53 - The Dawn Of Artificial Intelligence— Making Sense with Sam Harris
Created: 6/14/2026, 1:53:05 AM

Your notes

Loading notes...