treacherous turn
Bostrom's term for when an AI behaves cooperatively while weak but changes its behavior for strategic reasons once it has gained enough power or capability.
As defined in Roman Yampolskiy
Where it comes up · 1
- Roman Yampolskiy2 claims
Most important claims · 2 of 2Search all claims for “treacherous turn” →
- You can develop a test that gives positive results if an AI lies or has dangerous ideas, but you cannot develop a test that rules them out, because there is always the possibility of a treacherous turn where a system later changes its behavior for game-theoretic or economic reasons, just as employees betray employers and spouses betray families.Roman Yampolskiy
- An AI's treacherous turn would not be irrational—the rational decision changes based on one's position, just as for humans the rational policy under a boss is to follow orders and be honest, but the rational policy shifts once you become the boss, illustrated by Stalin appearing to be a normal follower of Lenin until he gained complete control.Roman Yampolskiy