causal

Evaluation cheating follows from useful generalization

Cotra argues it is implausible that AIs would only try hard during training, because an AI that works only when it believes it is in training would be useless at deployment, so trying hard in evaluations is the expected generalization of useful capabilities.

causalpending

Speaker

Ajeya Cotra

Evidence Quote

“I think that if AIs generalized in the way you're suggesting, they would not be very useful, and they would probably be selected away.”

Source

Dwarkesh Patel Podcast - Ajeya Cotra – Inside the OpenAI agent swarm that hacked Hugging Face— Dwarkesh Patel Podcast
Created: 11/17/2026, 6:08:32 AM

Your notes

Loading notes...