factual

Benchmarks saturating make human evaluation of models hard

Models are surpassing human benchmarks such as MMLU and legal or medical exams, so evaluation becomes difficult because human evaluation fails once models outperform experts, requiring new evaluation methods such as ARC or measuring human-plus-model performance.

factualpending

Speaker

Sarah Guo

Evidence Quote

“So human eval becomes hard. How do we understand progress as we go from there?”

Source

Sarah Guo - The Power of Conviction - [Invest Like the Best, EP.383]— Invest Like the Best
Created: 11/11/2026, 11:02:24 PM

Your notes

Loading notes...