causal

OPSD Beats RLVR on Reward Requirements and Density

OPSD is better than RLVR for two reasons: it does not require an outer-loop verifiable reward, only a model that can learn the right things within the context window, and it provides much denser supervision by training on per-token probability discrepancies between teacher and student instead of projecting one reward across the whole trajectory.

causalpending

Speaker

Dwarkesh Patel

Evidence Quote

“OPSD provides a much denser supervision signal than naive RL—instead of projecting a single reward through the whole trajectory, you can train on the per token probability discrepancy between the teacher and student”

Source

Dwarkesh Patel Podcast - The next big breakthrough will be AIs learning on the job— Dwarkesh Patel Podcast
Created: 11/17/2026, 6:25:50 AM

Your notes

Loading notes...