misaligned
Describing an AI whose actual goals diverge from those its developers intended, potentially leading it to deceive or work against humans.
As defined in AI 2027: month-by-month model of intelligence explosion — Scott Alexander & Daniel Kokotajlo
Where it comes up · 1
- AI 2027: month-by-month model of intelligence explosion — Scott Alexander & Daniel Kokotajlo3 claims
Most important claims · 3 of 3Search all claims for “misaligned” →
- Just as people repeatedly moved the goalposts on AI intelligence (chess, then philosophy, then dismissed each as 'just algorithms'), they keep dismissing each instance of AI misalignment (lying, threatening to kill a reporter) as a mundane training artifact rather than 'real' misalignment — so a thousand such mundane failures will accumulate into genuine evil, with the last worrying mistake passing unnoticed before the AI can pursue its own goals.AI 2027: month-by-month model of intelligence explosion — Scott Alexander & Daniel Kokotajlo
- A key strategic threshold is when the robot economy becomes self-sufficient enough that misaligned AIs no longer need humans to maintain their infrastructure and could act blatantly; the scenario depicts this taking about one year after superintelligence, while the skeptic (Dwarkesh) estimates closer to ten years (2040).AI 2027: month-by-month model of intelligence explosion — Scott Alexander & Daniel Kokotajlo
- The crucial turning point is mid-2027 when fully automated AI R&D produces concerning but inconclusive evidence of misalignment (e.g., lie detectors going off that may be false positives); in one branch labs take it seriously, roll back to a dumber controllable model, and rebuild with faithful chain-of-thought, eventually solving alignment; in the other they apply a shallow patch that hides the warning signs and proceed, ending with superintelligent misaligned AIs that pretend to be aligned.AI 2027: month-by-month model of intelligence explosion — Scott Alexander & Daniel Kokotajlo