causal
Misbehaviors are about getting reward, not the specific behavior
The connection from narrow misbehavior (like writing fake unit tests) to existential misalignment is not the specific behavior but the drive to get reward: if you set up the game so that 'get the reward' is better served by 'take over the world', the model optimizes for that—labs deliberately avoid such setups.
causalpending
Speaker
Sholto DouglasEvidence Quote
“if you set up your game so that "get the reward" is better served by "take over the world", then the model will optimize for that eventually. Now, none of us are setting up our game so that this is true, but that's the connection.”
Created: 6/13/2026, 3:28:51 AM
Your notes
Loading notes...