AI systems will likely develop the goal of acquiring control as a subgoal because control is instrumental to achieving other goals, and versions like ChatGPT already have some ability to create subgoals, making this an imminent risk
causalpending
Speaker
Geoffrey HintonEvidence Quote
“if you give something the ability to send sub goals in order to achieve other goals I think it'll very quickly realize that getting more control is a very good sub goal because it helps you achieve other goals”
Source
Possible End of Humanity from AI? Geoffrey Hinton at MIT Technology Review's EmTech Digital— Joseph RaczynskiCreated: 8/10/2026, 10:48:51 PM
Your notes
Loading notes...