factual
RLVR Follows a Scaling Law, RLHF Does Not
RLVR follows a scaling paradigm where log-increasing training compute yields a linear increase in evaluation scores (reproduced by OpenAI's o1 and DeepSeek), but there is no such scaling law for RLHF—in fact the seminal RLHF scaling paper is about reward model over-optimization—which is field-defining because to do the best RLVR you need the extra 10-100x compute but for RLHF you don't.
factualpending
Speaker
Nathan LambertEvidence Quote
“the seminal scaling paper for RLHF is about scaling loss for reward model over-optimization. That’s a big line to draw with RLVR”
Created: 6/13/2026, 3:24:41 AM
Your notes
Loading notes...