causal
OPSD Beats RLVR on Reward Requirements and Density
OPSD is better than RLVR for two reasons: it does not require an outer-loop verifiable reward, only a model that can learn the right things within the context window, and it provides much denser supervision by training on per-token probability discrepancies between teacher and student instead of projecting one reward across the whole trajectory.
causalpending
Speaker
Dwarkesh PatelEvidence Quote
“OPSD provides a much denser supervision signal than naive RL—instead of projecting a single reward through the whole trajectory, you can train on the per token probability discrepancy between the teacher and student”
Source
Dwarkesh Patel Podcast - The next big breakthrough will be AIs learning on the job— Dwarkesh Patel PodcastCreated: 11/17/2026, 6:25:50 AM
Your notes
Loading notes...