normative

Values Are Constructed, Not Instructions

Under moral constructivism, values are constructed by biological creatures with feelings and preferences as systematizations of underlying moral intuitions; since LLMs have no such intuitions, instructing an AI (e.g., to make paperclips) is not giving it a value, and calling AI safety work 'value alignment' mistakes the concept at step zero.

normativepending

Speaker

Sean Carroll

Evidence Quote

“Moral constructivism is the idea that morals are not objectively out there in the world, nor are they just completely made up or arbitrary. They are constructed by human beings for certain definable reasons... LLMs have no underlying moral intuitions and inclinations, and giving them instructions is not the same thing.”

Source

258 | Solo: AI Thinks Different— Sean Carroll's Mindscape
Created: 6/13/2026, 12:12:00 AM

Your notes

Loading notes...