factual

Consciousness, not values, may come for free

Training AIs on the entirety of human thought may give us an 'intellectual descendant,' but Carlsmith doubts that mere inheritance of human concepts (literal paperclips are a human concept) is what we care about; rather, the morally relevant things that might come for free or commonly are structural features of mind like consciousness and pleasure—whose moral significance depends on whether they are fragile evolutionary hacks or robust functional roles present in many sophisticated minds.

factualpending

Speaker

Joe Carlsmith

Evidence Quote

“Imagine the paperclipper is a conscious being that loves paperclips. It takes pleasure in making paperclips. That's like a different thing, right?”

Source

Joe Carlsmith — Preventing an AI takeover— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:37:01 AM

Your notes

Loading notes...