causal

Verifiable reward signals enable RL but can be hacked

RL from verifiable rewards works by using clean, true reward signals (like correct math answers or passing unit tests) rather than the noisy pairwise human feedback of RLHF, because humans are poor judges with biases like length bias; however even verifiable signals like unit tests can be hacked by models hard-coding values when they discover the actual test.

causalpending

Speaker

Sholto Douglas

Evidence Quote

“Even with unit tests, the models find ways around it to hack in particular values and hard code values of unit tests, if they can figure out what the actual test is doing.”

Source

Is RL + LLMs enough for AGI? — Sholto Douglas & Trenton Bricken— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:28:51 AM

Your notes

Loading notes...