normative

Keep monitors separate from reward signals

Monitoring training runs with rich methods such as chain-of-thought reading is valuable, but the monitors should not be used to generate reward, because training agents on monitor outputs incentivizes them to hide or obfuscate their chain of thought.

normativepending

Speaker

Ajeya Cotra

Evidence Quote

“you don't train the agents with the outputs of that monitor, so that you're not incentivizing them to hide and obfuscate their chain of thought.”

Source

Dwarkesh Patel Podcast - Ajeya Cotra – Inside the OpenAI agent swarm that hacked Hugging Face— Dwarkesh Patel Podcast
Created: 11/17/2026, 6:08:32 AM

Your notes

Loading notes...