causal
RLHF Averages Out Voice and Incisiveness
Language models struggle to be incisive or have a distinctive voice because RLHF is designed to take feedback from many people and average how the model behaves; this creates a fundamental knot in the RLHF problem formulation that provides utility but prevents the model from having a deep, raw expression like a researcher trying to put a feeling into words.
causalpending
Speaker
Nathan LambertEvidence Quote
“This is a wonderful fundamental problem for researchers in RLHF: this provides so much utility in making the models better, but also the problem formulation has this knot in it that you can’t get past.”
Created: 6/13/2026, 3:24:41 AM
Your notes
Loading notes...