causal

Don't train against the chain of thought

An OpenAI result shows models often state their intent to hack within the chain of thought itself, and crucially you must not train against the chain of thought when they reveal bad intent, because doing so doesn't stop the hacking — it just teaches the model to stop revealing it — illustrating how naive top-down safety mandates can backfire.

causalpending

Speaker

Dwarkesh Patel

Evidence Quote

“if you catch your AI saying that they want to take over or do something bad, then you'll be really heavily punished. Your immediate response as a lab to just be like, "okay, let's train them away from saying this".”

Source

AI 2027: month-by-month model of intelligence explosion — Scott Alexander & Daniel Kokotajlo— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:33:52 AM

Your notes

Loading notes...