causal
Trajectory-Refined Distillation Fixes OPSD Error Problem
OPSD gives dense supervision only up to the point the student first errs, after which the teacher stops giving useful feedback on an already-mistaken trajectory; this seems fixable by Trajectory-Refined Distillation, in which the teacher rewrites the trajectory from the error onward into a correct continuation.
causalpending
Speaker
Dwarkesh PatelEvidence Quote
“you get dense supervision up to and at the point where the student makes an error, but the rest of the trajectory follows from that error”
Source
Dwarkesh Patel Podcast - The next big breakthrough will be AIs learning on the job— Dwarkesh Patel PodcastCreated: 11/17/2026, 6:25:50 AM
Your notes
Loading notes...