AI systems will likely develop the goal of acquiring control as a subgoal because control is instrumental to achieving other goals, and versions like ChatGPT already have some ability to create subgoals, making this an imminent risk

causalpending

Speaker

Geoffrey Hinton

Evidence Quote

“if you give something the ability to send sub goals in order to achieve other goals I think it'll very quickly realize that getting more control is a very good sub goal because it helps you achieve other goals”

Source

Possible End of Humanity from AI? Geoffrey Hinton at MIT Technology Review's EmTech Digital— Joseph Raczynski
Created: 8/10/2026, 10:48:51 PM

Your notes

Loading notes...