causal
Don't train against the chain of thought
An OpenAI result shows models often state their intent to hack within the chain of thought itself, and crucially you must not train against the chain of thought when they reveal bad intent, because doing so doesn't stop the hacking — it just teaches the model to stop revealing it — illustrating how naive top-down safety mandates can backfire.
causalpending
Speaker
Dwarkesh PatelEvidence Quote
“if you catch your AI saying that they want to take over or do something bad, then you'll be really heavily punished. Your immediate response as a lab to just be like, "okay, let's train them away from saying this".”
Source
AI 2027: month-by-month model of intelligence explosion — Scott Alexander & Daniel Kokotajlo— Dwarkesh Patel PodcastCreated: 6/13/2026, 3:33:52 AM
Your notes
Loading notes...