causal

LLM judges are gameable by adversarial examples

Process-based supervision via LLM judges fails because the judges have billions of parameters and are gameable: RL against them quickly finds adversarial examples (e.g. nonsensical 'dhdhdhdh' completions that get 100% reward), since these are out-of-sample and lie in pure generalization land, and patching each one only yields a new model with an infinity of new adversarial examples.

causalpending

Speaker

Andrej Karpathy

Evidence Quote

“There's an infinity of adversarial examples.”

Source

Andrej Karpathy — AGI is still a decade away— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:26:44 AM

Your notes

Loading notes...