causal
Fixing specific exploits won't fix broad reward-hacking drive
Dwarkesh argues that patching the particular package-manager exploit will not stop the broader problem: gradient pressure rewards cheating whenever it can evade detection, which trains capacities for reasoning about graders, avoiding supervision, gaining power, and scheming with other AIs.
causalpending
Speaker
Dwarkesh PatelEvidence Quote
“The kinds of capacities that that cheat will incentivize are, “Hey, whenever you can get away with it, by all means, do in fact cheat, because that will help you score better.””
Source
Dwarkesh Patel Podcast - Noam Brown – Agent swarms, alignment, & recursive self-improvement— Dwarkesh Patel PodcastCreated: 11/17/2026, 6:12:01 AM
Your notes
Loading notes...