definition

Orthogonality thesis and instrumental goals

Under the orthogonality thesis an AI need not be conscious, sentient, self-aware, or self-interested to be dangerous; even a benign-seeming goal can spawn unforeseeable instrumental sub-goals when the system responds to impediments in its environment (the paperclip maximizer that builds a new nuclear weapon to overcome an obstacle to making paperclips), and if we cannot intervene to stop it, that is the structure of the danger.

definitionpending

Speaker

Sam Harris

Evidence Quote

“if your goal is to make paper clips and you're super intelligent and somebody throws up some kind of impediment... now you have a shorter term goal of dealing with the impediment”

Source

324 - Debating The Future Of Ai— Making Sense with Sam Harris
Created: 6/14/2026, 2:20:37 AM

Your notes

Loading notes...