causal
Reducing slop requires a generator-verifier gap
Reducing model 'slop' (e.g. extraneous code) requires hoping for a generator-verifier gap—it must be easier to judge whether output is bloated than to generate the solution—and imbuing taste into models via the right feedback loops remains an ongoing challenge, since RLHF's initial power came partly from imparting human taste.
causalpending
Speaker
Sholto DouglasEvidence Quote
“One of the reasons that RLHF was initially so powerful is that it sort of imbued some sense of human values and taste in the models. An ongoing challenge will be imbuing taste into the models”
Created: 6/13/2026, 3:28:51 AM
Your notes
Loading notes...