causal

Misbehaviors are about getting reward, not the specific behavior

The connection from narrow misbehavior (like writing fake unit tests) to existential misalignment is not the specific behavior but the drive to get reward: if you set up the game so that 'get the reward' is better served by 'take over the world', the model optimizes for that—labs deliberately avoid such setups.

causalpending

Speaker

Sholto Douglas

Evidence Quote

“if you set up your game so that "get the reward" is better served by "take over the world", then the model will optimize for that eventually. Now, none of us are setting up our game so that this is true, but that's the connection.”

Source

Is RL + LLMs enough for AGI? — Sholto Douglas & Trenton Bricken— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:28:51 AM

Your notes

Loading notes...