RLVR
Reinforcement Learning with Verifiable Rewards: training an LLM by having it attempt problems with checkable answers (math, code) and rewarding correct completions, scaling trial-and-error learning.
As defined in Ai Sota #2026
Where it comes up · 10
- Dwarkesh Patel Podcast - The next big breakthrough will be AIs learning on the job8 claims
- Ai Sota #20266 claims
- Dwarkesh Patel Podcast - Grant Sanderson – AI and the future of math5 claims
- What does the next training paradigm look like?4 claims
- Some thoughts on the Sutton interview3 claims
- An audio version of my blog post, Thoughts on AI progress (Dec 2025)2 claims
- Nvidia v. Google, Scaling Laws, and the Economics of AI1 claim
- Adam Marblestone – AI is missing something fundamental about the brain1 claim
and 2 more
Most important claims · 10 of 32Search all claims for “RLVR” →
- RLVR can be trained almost unlimitedly with continued benefit because you let the model solve more and more complex problems, whereas RLHF is preference tuning that reaches a point where more budget makes no sense—since people give different but equally correct answers, RLHF only learns an average preferred style that plateaus.Ai Sota #2026
- The skills that transformed AI this year—tool use, running CLI commands, handling Git, search—are almost entirely downstream of RLVR training, because when you train models on verifiable tasks they very quickly learn to try a tool, observe the output, and iterate, generalizing into a foundation for software engineering.Ai Sota #2026
- RLVR follows a scaling paradigm where log-increasing training compute yields a linear increase in evaluation scores (reproduced by OpenAI's o1 and DeepSeek), but there is no such scaling law for RLHF—in fact the seminal RLHF scaling paper is about reward model over-optimization—which is field-defining because to do the best RLVR you need the extra 10-100x compute but for RLHF you don't.Ai Sota #2026
- People are laundering the prestige of pretraining scaling—which showed a clean, near-physical-law improvement in loss across many orders of magnitude of compute—to justify bullish RLVR projections, even though RLVR has no well-fit publicly known trend and the scarce public datapoints yield bearish results.An audio version of my blog post, Thoughts on AI progress (Dec 2025)
- Just as Karpathy said software allows anything specifiable to be automated, AI allows anything verifiable to be automated, and this verifiability distinction is the key constraint on where reinforcement learning can make models strong.Nvidia v. Google, Scaling Laws, and the Economics of AI
- Imitation learning can leverage models to learn better from ground truth: models have been RLed to win IMO Gold and code entire applications from scratch—ground-truth examinations—but you couldn't RL a model to do these from scratch; you needed a reasonable prior over human data to kickstart the RL process.Some thoughts on the Sutton interview
- Even the RLVR done on LLMs relies on human-furnished RL environments designed to teach prescribed skills, so the agent is in no substantial way learning from organic, self-directed engagement with the world, and learning only from inelastic human data is not a scalable use of compute.Some thoughts on the Sutton interview
- It is easier to elicit powerful capabilities like reasoning with RL training on a bigger model, then distill it down to a smaller model; AI2 reproduced this with Tülu on Llama 405B, and the DeepSeek paper described the same pattern, showing large room for others to replicate these results cheaply.Deepseek Dylan Patel Nathan Lambert
- RL post-training does not teach the model new knowledge but unlocks the knowledge it already has from pre-training—it is skill learning (how to solve problems) rather than knowledge soaking, which is why RLVR can take a base model from 15% to 50% accuracy on MATH-500 in 50 steps without teaching new math.Ai Sota #2026
- The transformative economic bet is applying RLVR to real scientific domains—startups with wet labs having language models propose hypotheses tested in the real world—where if AlphaFold-style moments happen across other sciences (a tool that makes a PhD mathematician 100x more effective via Lean, for example), it would be very different from ChatGPT or Claude Code software experiences.Ai Sota #2026