factual

Naive Fine-Tuning On Observed Tokens Fails

If you tried to fit the transition-model spec into LLMs by fine-tuning on all observed tokens, in practice the most naive way of doing this doesn't work well, according to researcher friends.

factualpending

Speaker

Dwarkesh Patel

Evidence Quote

“in practice the most naive way of doing this doesn’t work well.”

Source

Some thoughts on the Sutton interview— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:28:01 AM

Your notes

Loading notes...