definition

There is no pre-normative notion of reflection

There is no off-the-shelf, pre-normative process of 'reflection' that you can run an agent through to read off its values; in reality there is a whole pattern of empirical facts about what happens when you put an agent through various processes, and you have to take a substantive normative stand on which development processes preserve what you care about versus which take it off the rails—a stand that itself requires endorsing certain meta-processes.

definitionpending

Speaker

Joe Carlsmith

Evidence Quote

“you have to look at the pattern of outputs and be like, "Okay, what do I make of that?"”

Source

Joe Carlsmith — Preventing an AI takeover— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:37:01 AM

Your notes

Loading notes...