causal

Researchers reward-hack by optimizing for evals

Because RL training requires choosing which environments to train on, researchers inadvertently take inspiration from the evals they want to look good on, effectively making the human researchers the real reward hackers; combined with inadequate generalization this explains the gap between eval and real-world performance.

causalpending

Speaker

Ilya Sutskever

Evidence Quote

“people take inspiration from the evals. ... I think that is something that happens, and it could explain a lot of what's going on.”

Source

Ilya Sutskever – We're moving from the age of scaling to the age of research— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:24:31 AM

Your notes

Loading notes...