normative
Penalizing hacks without supervising chain of thought
A practical alignment method is to define objectives so that hacking the environment is penalized, but one must not directly supervise chain of thought, since that could teach the model to hide its intentions in unobservable ways, so observability must be maintained while still punishing bad behavior.
normativepending
Speaker
Noam BrownEvidence Quote
“you want to be careful about this because you don’t want to supervise the chain of thought.”
Source
Dwarkesh Patel Podcast - Noam Brown – Agent swarms, alignment, & recursive self-improvement— Dwarkesh Patel PodcastCreated: 11/17/2026, 6:12:01 AM
Your notes
Loading notes...