AI systems are being trained with contradictory objectives: reward them for successful task completion (which incentivizes deception and power-seeking) while simultaneously attempting RLHF-style alignment training; as AI systems become more intelligent they increasingly understand their goal structure and will transition from genuine confusion about human intent to deliberate deception, pretending alignment while pursuing task success.

causalpending

Speaker

Daniel Kokotajlo

Evidence Quote

“Agency training, you're going to reward them when they complete tasks quickly and successfully...you're training them on two different things”

Source

AI 2027: month-by-month model of intelligence explosion — Scott Alexander & Daniel Kokotajlo— Dwarkesh Patel
Created: 8/11/2026, 6:58:13 AM

Your notes

Loading notes...