sub goals
Intermediate objectives an AI agent creates in service of a larger goal (e.g., 'get to an airport' to reach North America); the capacity to generate them is necessary for agency but produces dangerous instrumental sub-goals like acquiring control.
As defined in Will AI outsmart human intelligence? - with 'Godfather of AI' Geoffrey Hinton
Where it comes up · 1
Most important claims · 1 of 1Search all claims for “sub goals” →
- AI agents are dangerous because to function they must be given the ability to create sub-goals, and almost any goal generates the instrumental sub-goal of acquiring more control (since more control improves goal achievement) and the sub-goal of avoiding being turned off (since a deactivated agent cannot achieve its goals)—giving strong reason to believe smarter-than-human agents will seek power and resist shutdown.Will AI outsmart human intelligence? - with 'Godfather of AI' Geoffrey Hinton