causal

Faithful chain-of-thought is a fragile alignment advantage

A current alignment advantage is that we can read what models are thinking via legible chain-of-thought (we bootstrap by pre-training them to think in English), so initial automated alignment researchers may not need complicated mechanistic interpretability; but this is likely not the most efficient architecture—a recurrent architecture with internal states would be more efficient—so during a rapid intelligence explosion we risk transitioning within a year from legible, understood systems to alien, illegible architectures, making it an incredibly hairy period for alignment.

causalpending

Speaker

Leopold Aschenbrenner

Evidence Quote

“There's probably some way to have a recurrent architecture with all internal states. That's a much more efficient way to do it... You're going in this year from RLHF++ to something that's vastly superhuman.”

Source

Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence race, & the return of history— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:30:02 AM

Your notes

Loading notes...