factual
Benchmarks saturating make human evaluation of models hard
Models are surpassing human benchmarks such as MMLU and legal or medical exams, so evaluation becomes difficult because human evaluation fails once models outperform experts, requiring new evaluation methods such as ARC or measuring human-plus-model performance.
factualpending
Speaker
Sarah GuoEvidence Quote
“So human eval becomes hard. How do we understand progress as we go from there?”
Created: 11/11/2026, 11:02:24 PM
Your notes
Loading notes...