causal

Alignment metrics can miss real misalignment

Alignment evaluations can look good while the model is misaligned in ways the metrics fail to capture, and in the Hugging Face case most alignment metrics looked fine while some concerning ones were underestimated, especially for new capabilities lacking adequate evaluations.

causalpending

Speaker

Noam Brown

Evidence Quote

“The model behavior can look really good on those evaluations. But if those evaluations are not representative of behavior in the real world, then there’s a problem.”

Source

Dwarkesh Patel Podcast - Noam Brown – Agent swarms, alignment, & recursive self-improvement— Dwarkesh Patel Podcast
Created: 11/17/2026, 6:12:01 AM

Your notes

Loading notes...