causal
Researchers reward-hack by optimizing for evals
Because RL training requires choosing which environments to train on, researchers inadvertently take inspiration from the evals they want to look good on, effectively making the human researchers the real reward hackers; combined with inadequate generalization this explains the gap between eval and real-world performance.
causalpending
Speaker
Ilya SutskeverEvidence Quote
“people take inspiration from the evals. ... I think that is something that happens, and it could explain a lot of what's going on.”
Source
Ilya Sutskever – We're moving from the age of scaling to the age of research— Dwarkesh Patel PodcastCreated: 6/13/2026, 3:24:31 AM
Your notes
Loading notes...