causal

RLHF Fine-Tuning Reduces Bullshitting

Chatbots like ChatGPT take a pretrained language model and add fine-tuning via reinforcement learning from human rankings of outputs (by helpfulness, harmlessness, honesty), making them less toxic and less prone to 'bullshitting' in Harry Frankfurt's sense—speaking without regard for truth—than vanilla models like GPT-3.

causalpending

Speaker

Raphael Milliere

Evidence Quote

“after you've done that, you get something like ChatGPT that is generally a less toxic model than a vanilla large pretrained language model, is less prone to just outputting random made up facts”

Source

230 | Raphaël Millière on How Artificial Intelligence Thinks— Sean Carroll's Mindscape
Created: 6/13/2026, 12:19:51 AM

Your notes

Loading notes...