normative

Benchmarks are a poor measure of progress

Benchmark scores are a terrible way to assess the overall rate of AI progress, and smoother internal metrics such as pre-training perplexity and RL compute efficiency give a better read on how fast the field is moving.

normativepending

Speaker

Paul Christiano

Evidence Quote

“Benchmark scores are a terrible way to assess the overall rate of progress. There are a bunch of much more smoothly varying metrics recorded internally.”

Source

Dwarkesh Patel Podcast - Paul Christiano – We could be 12 months away from AIs that can overthrow the government— Dwarkesh Patel Podcast
Created: 11/17/2026, 6:22:08 AM

Your notes

Loading notes...