Sheela
Sheela
Home
Add
Questions
Thinkers
Claims
People
Concepts
Jargon
Sources
Health
Predictions
Collections
Domain Landscape
Contribute
Review
Dedup
Topic requests
Settings
Help

← Jargon

Jargontechnology· 2 claims· 1 episode

treacherous turn

Bostrom's term for when an AI behaves cooperatively while weak but changes its behavior for strategic reasons once it has gained enough power or capability.

As defined in Roman Yampolskiy


Where it comes up · 1
  • Roman Yampolskiy2 claims

Most important claims · 2 of 2Search all claims for “treacherous turn” →
  • You can develop a test that gives positive results if an AI lies or has dangerous ideas, but you cannot develop a test that rules them out, because there is always the possibility of a treacherous turn where a system later changes its behavior for game-theoretic or economic reasons, just as employees betray employers and spouses betray families.Roman Yampolskiy
  • An AI's treacherous turn would not be irrational—the rational decision changes based on one's position, just as for humans the rational policy under a boss is to follow orders and be honest, but the rational policy shifts once you become the boss, illustrated by Stalin appearing to be a normal follower of Lenin until he gained complete control.Roman Yampolskiy