normative
Benchmarks are a poor measure of progress
Benchmark scores are a terrible way to assess the overall rate of AI progress, and smoother internal metrics such as pre-training perplexity and RL compute efficiency give a better read on how fast the field is moving.
normativepending
Speaker
Paul ChristianoEvidence Quote
“Benchmark scores are a terrible way to assess the overall rate of progress. There are a bunch of much more smoothly varying metrics recorded internally.”
Source
Dwarkesh Patel Podcast - Paul Christiano – We could be 12 months away from AIs that can overthrow the government— Dwarkesh Patel PodcastCreated: 11/17/2026, 6:22:08 AM
Your notes
Loading notes...