causal
RLHF Fine-Tuning Reduces Bullshitting
Chatbots like ChatGPT take a pretrained language model and add fine-tuning via reinforcement learning from human rankings of outputs (by helpfulness, harmlessness, honesty), making them less toxic and less prone to 'bullshitting' in Harry Frankfurt's sense—speaking without regard for truth—than vanilla models like GPT-3.
causalpending
Speaker
Raphael MilliereEvidence Quote
“after you've done that, you get something like ChatGPT that is generally a less toxic model than a vanilla large pretrained language model, is less prone to just outputting random made up facts”
Created: 6/13/2026, 12:19:51 AM
Your notes
Loading notes...