causal

Reducing slop requires a generator-verifier gap

Reducing model 'slop' (e.g. extraneous code) requires hoping for a generator-verifier gap—it must be easier to judge whether output is bloated than to generate the solution—and imbuing taste into models via the right feedback loops remains an ongoing challenge, since RLHF's initial power came partly from imparting human taste.

causalpending

Speaker

Sholto Douglas

Evidence Quote

“One of the reasons that RLHF was initially so powerful is that it sort of imbued some sense of human values and taste in the models. An ongoing challenge will be imbuing taste into the models”

Source

Is RL + LLMs enough for AGI? — Sholto Douglas & Trenton Bricken— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:28:51 AM

Your notes

Loading notes...