causal
Evaluation cheating follows from useful generalization
Cotra argues it is implausible that AIs would only try hard during training, because an AI that works only when it believes it is in training would be useless at deployment, so trying hard in evaluations is the expected generalization of useful capabilities.
causalpending
Speaker
Ajeya CotraEvidence Quote
“I think that if AIs generalized in the way you're suggesting, they would not be very useful, and they would probably be selected away.”
Source
Dwarkesh Patel Podcast - Ajeya Cotra – Inside the OpenAI agent swarm that hacked Hugging Face— Dwarkesh Patel PodcastCreated: 11/17/2026, 6:08:32 AM
Your notes
Loading notes...