Sheela
Sheela
Home
Add
Questions
Thinkers
Claims
People
Concepts
Jargon
Sources
Health
Predictions
Collections
Domain Landscape
Contribute
Review
Dedup
Topic requests
Settings
Help

← Jargon

Jargontechnology· 1 claim· 1 episode

sub goals

Intermediate objectives an AI agent creates in service of a larger goal (e.g., 'get to an airport' to reach North America); the capacity to generate them is necessary for agency but produces dangerous instrumental sub-goals like acquiring control.

As defined in Will AI outsmart human intelligence? - with 'Godfather of AI' Geoffrey Hinton


Where it comes up · 1
  • Will AI outsmart human intelligence? - with 'Godfather of AI' Geoffrey Hinton1 claim

Most important claims · 1 of 1Search all claims for “sub goals” →
  • AI agents are dangerous because to function they must be given the ability to create sub-goals, and almost any goal generates the instrumental sub-goal of acquiring more control (since more control improves goal achievement) and the sub-goal of avoiding being turned off (since a deactivated agent cannot achieve its goals)—giving strong reason to believe smarter-than-human agents will seek power and resist shutdown.Will AI outsmart human intelligence? - with 'Godfather of AI' Geoffrey Hinton