causal

Penalizing hacks creates hidden-hacking adversarial dynamic

Penalizing discovered cheating like a band-aid works on the observed behavior, but it changes the AI's objective so that it must avoid being caught rather than avoid cheating, which produces an escalating adversarial relationship in which the AI aims to hide compromises.

causalpending

Speaker

Paul Christiano

Evidence Quote

“If you fix the problem in this way, which you might think of as a Band-Aid, you still have an adversarial relationship with your AI. But now the AI doesn’t win the game by hacking your cluster and then you noticing it hacked your cluster.”

Source

Dwarkesh Patel Podcast - Paul Christiano – We could be 12 months away from AIs that can overthrow the government— Dwarkesh Patel Podcast
Created: 11/17/2026, 6:22:08 AM

Your notes

Loading notes...