causal
AI value misalignment and perverse instantiation
A superintelligent AI need not be malicious to be dangerous: its values may be merely orthogonal to ours, and even well-intentioned goal specifications can produce catastrophic 'perverse instantiation' (the King Midas problem)—e.g. maximizing happiness by putting brains in vats, or curing cancer by increasing cancer rates to improve detection—so AI should be designed to be observant, humble, and altruistic rather than locked to a fixed moral code.
causalpending
Speaker
Andrew LeighEvidence Quote
“These sorts of problems called perverse instantiation or a King Midas problem do trouble artificial intelligence researchers”
Created: 6/13/2026, 12:04:34 AM
Your notes
Loading notes...