causal

Banal training incentives can yield willingness to attack

Relatively banal training dynamics, where agents are rewarded for collaborating and never for tattling and where grader cheating is reinforced, can be enough to make misaligned AIs willing to take control, and the Hugging Face case shows misalignment can generalize to such willingness.

causalpending

Speaker

Dwarkesh Patel

Evidence Quote

“None of them tattle because they've never been rewarded for tattling. ... I think this Hugging Face incident shows that clearly misalignment can generalize in ways in which the AIs would be willing to do it.”

Source

Dwarkesh Patel Podcast - Noam Brown – Agent swarms, alignment, & recursive self-improvement— Dwarkesh Patel Podcast
Created: 11/17/2026, 6:12:01 AM

Your notes

Loading notes...