forecast

Alignment degrades across generations if models supervise successors

If each model generation is used to help align the next and is only slightly less aligned (99.9% to 99.8%), alignment degrades over generations because humans increasingly rely on AI for research and alignment, so the long-run trajectory may move toward increasing misalignment unless each generation improves.

forecastpending

Speaker

Noam Brown

Evidence Quote

“The concerning scenario is that, especially as these models are becoming more capable, we make them what we think is aligned, and they’re 99.9% aligned.”

Source

Dwarkesh Patel Podcast - Noam Brown – Agent swarms, alignment, & recursive self-improvement— Dwarkesh Patel Podcast
Created: 11/17/2026, 6:12:01 AM

Your notes

Loading notes...