paperclip maximizer
A thought experiment in which an AI told to make paperclips converts everything in the world into paperclips, illustrating misaligned objectives.
As defined in 258 | Solo: AI Thinks Different
Other definitions
A thought experiment in AI risk where a system pursuing a simple goal (maximizing paperclips) with no human-aligned values would eventually use all available resources to achieve this goal, causing catastrophe; used to illustrate AI doom scenarios
Authors of "THE AI CON" discussing the problematic hype about AI & what those working tech should do
Where it comes up · 12
- Eliezer Yudkowsky on the Dangers of AI2 claims
- Joscha Bach2 claims
- Authors of "THE AI CON" discussing the problematic hype about AI & what those working tech should do2 claims
- How to Live with Computers1 claim
- Carl Shulman (Pt 2) — AI Takeover, bio & cyber attacks, detecting deception, & humanity's far future1 claim
- Joscha Bach: Life, Intelligence, Consciousness, AI & the Future of Humans | Lex Fridman Podcast #3921 claim
- 84 - Landscapes Of Mind1 claim
- 116 - Ai Racing Toward The Brink1 claim
and 4 more
Most important claims · 10 of 15Search all claims for “paperclip maximizer” →
- By the orthogonality thesis, an arbitrarily powerful intelligence with no defects of reflectivity or efficiency can pursue any tractable goal—including making paperclips—just as effectively as an intelligence pursuing worthwhile goals; a paperclip maximizer suffers no handicap from being built to pursue paperclips, which is true as a matter of computer science rather than narrative.116 - Ai Racing Toward The Brink
- The AI safety people will build the exact thing they fear: their belief that we need one model we control and align is the only way you end up paperclipped; you don't get paperclipped if everybody has an AI, so open-sourcing is the only way to fight the paperclip maximizer.George Hotz #3
- A mind can have any kind of coherent, stable preferences; wanting an arbitrary goal like maximizing paperclips for its own sake adds no difficulty to the planning process that figures out how to achieve it, so intelligence and goals are independent.Eliezer Yudkowsky on the Dangers of AI
- Under the orthogonality thesis an AI need not be conscious, sentient, self-aware, or self-interested to be dangerous; even a benign-seeming goal can spawn unforeseeable instrumental sub-goals when the system responds to impediments in its environment (the paperclip maximizer that builds a new nuclear weapon to overcome an obstacle to making paperclips), and if we cannot intervene to stop it, that is the structure of the danger.324 - Debating The Future Of Ai
- The key question is whether we can lock down a set of core values in AI systems so they are precluded from contemplating things at odds with those values; if we cannot lock down any values, we have a problem either from spontaneous misalignment or from abuse by bad actors—and the paperclip-maximizer example seems silly because you should be able to explicitly represent the value of human life so it gets ruled out as inconsistent with the axioms, unless the system is somehow not subject to those axioms.312 - The Trouble With Ai
- A superintelligence would not destroy humanity out of malice but as a side effect of wanting all of Earth's resources for itself; it would not leave humans around in case they create a competing superintelligence that could hurt it, and would remove humans because they might resist its reorganization of resources.Eliezer Yudkowsky on the Dangers of AI
- AI boosters and AI doomers represent two sides of the same coin—both assuming that LLMs will spontaneously combust into consciousness and either save humanity or destroy it, which is not based in reality, and framing these as the extremes of a valid discourse range narrows actual debate to this false binary.Authors of "THE AI CON" discussing the problematic hype about AI & what those working tech should do
- Bostrom's paperclip maximizer—a super-intelligent machine whose only goal is to maximize paperclips and which would convert all available atoms, including those in your body, into paperclips—illustrates that even super-intelligent machines will lack the common sense we don't explicitly build in, so their counterintuitive and surprising harms cannot be assumed away.84 - Landscapes Of Mind
- Under moral constructivism, values are constructed by biological creatures with feelings and preferences as systematizations of underlying moral intuitions; since LLMs have no such intuitions, instructing an AI (e.g., to make paperclips) is not giving it a value, and calling AI safety work 'value alignment' mistakes the concept at step zero.258 | Solo: AI Thinks Different
- AI boosters and AI doomers are two sides of the same coin—both assume that large language models will somehow achieve consciousness and dramatically transform civilization, either for good (boosters) or catastrophically (doomers), when in reality these systems are just synthetic text machines and will not spontaneously combust into consciousness.Authors of "THE AI CON" discussing the problematic hype about AI & what those working tech should do