factual
Consciousness, not values, may come for free
Training AIs on the entirety of human thought may give us an 'intellectual descendant,' but Carlsmith doubts that mere inheritance of human concepts (literal paperclips are a human concept) is what we care about; rather, the morally relevant things that might come for free or commonly are structural features of mind like consciousness and pleasure—whose moral significance depends on whether they are fragile evolutionary hacks or robust functional roles present in many sophisticated minds.
factualpending
Speaker
Joe CarlsmithEvidence Quote
“Imagine the paperclipper is a conscious being that loves paperclips. It takes pleasure in making paperclips. That's like a different thing, right?”
Created: 6/13/2026, 3:37:01 AM
Your notes
Loading notes...