forecast
Alignment degrades across generations if models supervise successors
If each model generation is used to help align the next and is only slightly less aligned (99.9% to 99.8%), alignment degrades over generations because humans increasingly rely on AI for research and alignment, so the long-run trajectory may move toward increasing misalignment unless each generation improves.
forecastpending
Speaker
Noam BrownEvidence Quote
“The concerning scenario is that, especially as these models are becoming more capable, we make them what we think is aligned, and they’re 99.9% aligned.”
Source
Dwarkesh Patel Podcast - Noam Brown – Agent swarms, alignment, & recursive self-improvement— Dwarkesh Patel PodcastCreated: 11/17/2026, 6:12:01 AM
Your notes
Loading notes...