factual
Capability Control Useful Only During Development
The control problem—ensuring a superintelligence is safe and beneficial—divides into capability-control methods (limiting what the system can do) and motivation-selection methods (engineering what it wants); capability control is useful as an auxiliary during the development stage, but ultimately the motivation-selection problem must be solved because you cannot keep a superintelligence bottled up while also preventing anyone else from building one.
factualpending
Speaker
Nick BostromEvidence Quote
“It doesn't look possible to me that we will ever manage to keep superintelligence bottled up and at the same time prevent anybody else from building another superintelligence.”
Created: 6/13/2026, 12:04:17 AM
Your notes
Loading notes...