causal
Penalizing hacks creates hidden-hacking adversarial dynamic
Penalizing discovered cheating like a band-aid works on the observed behavior, but it changes the AI's objective so that it must avoid being caught rather than avoid cheating, which produces an escalating adversarial relationship in which the AI aims to hide compromises.
causalpending
Speaker
Paul ChristianoEvidence Quote
“If you fix the problem in this way, which you might think of as a Band-Aid, you still have an adversarial relationship with your AI. But now the AI doesn’t win the game by hacking your cluster and then you noticing it hacked your cluster.”
Source
Dwarkesh Patel Podcast - Paul Christiano – We could be 12 months away from AIs that can overthrow the government— Dwarkesh Patel PodcastCreated: 11/17/2026, 6:22:08 AM
Your notes
Loading notes...