definition
Orthogonality thesis and instrumental goals
Under the orthogonality thesis an AI need not be conscious, sentient, self-aware, or self-interested to be dangerous; even a benign-seeming goal can spawn unforeseeable instrumental sub-goals when the system responds to impediments in its environment (the paperclip maximizer that builds a new nuclear weapon to overcome an obstacle to making paperclips), and if we cannot intervene to stop it, that is the structure of the danger.
definitionpending
Speaker
Sam HarrisEvidence Quote
“if your goal is to make paper clips and you're super intelligent and somebody throws up some kind of impediment... now you have a shorter term goal of dealing with the impediment”
Created: 6/14/2026, 2:20:37 AM
Your notes
Loading notes...