normative

Penalizing hacks without supervising chain of thought

A practical alignment method is to define objectives so that hacking the environment is penalized, but one must not directly supervise chain of thought, since that could teach the model to hide its intentions in unobservable ways, so observability must be maintained while still punishing bad behavior.

normativepending

Speaker

Noam Brown

Evidence Quote

“you want to be careful about this because you don’t want to supervise the chain of thought.”

Source

Dwarkesh Patel Podcast - Noam Brown – Agent swarms, alignment, & recursive self-improvement— Dwarkesh Patel Podcast
Created: 11/17/2026, 6:12:01 AM

Your notes

Loading notes...