factual

RLVR Has No Hard Ceiling Unlike RLHF

RLVR can be trained almost unlimitedly with continued benefit because you let the model solve more and more complex problems, whereas RLHF is preference tuning that reaches a point where more budget makes no sense—since people give different but equally correct answers, RLHF only learns an average preferred style that plateaus.

factualpending

Speaker

Sebastian Raschka

Evidence Quote

“you learn that average preferred answer, and there’s no reason to keep training longer on it because it’s just a style.”

Source

Ai Sota #2026— Lex Fridman Podcast
Created: 6/13/2026, 3:24:41 AM

Your notes

Loading notes...