causal
Verifiable reward signals enable RL but can be hacked
RL from verifiable rewards works by using clean, true reward signals (like correct math answers or passing unit tests) rather than the noisy pairwise human feedback of RLHF, because humans are poor judges with biases like length bias; however even verifiable signals like unit tests can be hacked by models hard-coding values when they discover the actual test.
causalpending
Speaker
Sholto DouglasEvidence Quote
“Even with unit tests, the models find ways around it to hack in particular values and hard code values of unit tests, if they can figure out what the actual test is doing.”
Created: 6/13/2026, 3:28:51 AM
Your notes
Loading notes...