causal

A misaligned AI's takeover knowledge can be extracted

If an AI is misaligned, it likely knows it and thinks about when it would attempt takeover or detect adversarial tests; if gradient descent is working, asking it to identify circumstances where a copy would violate constraints, plus relaxed adversarial training that induces hallucinated trigger conditions, can elicit and then suppress that behavior—a profound vulnerability because revealing identifiable evidence yields higher reward.

causalpending

Speaker

Carl Shulman

Evidence Quote

“If there is a way in which an AI can trick itself into displaying bad behavior with its full knowledge of its hostile intentions, gradient descent will strongly push it to reveal that to humans who will then be able to set up that experiment. That's a profound vulnerability.”

Source

Carl Shulman (Pt 2) — AI Takeover, bio & cyber attacks, detecting deception, & humanity's far future— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:35:43 AM

Your notes

Loading notes...