factual
LLMs Trained Only to Predict Next Word
Although it feels like large language models plan whole documents, technically they are trained only to predict which word comes next; when trained on enough data this single objective produces fluent, coherent long-form text as if it had been planned.
factualpending
Speaker
Yejin ChoiEvidence Quote
“it's trained to predict which word comes next, but if you train it so well on so much data, then we realize that, "Wow, it can actually generate very nice fluent long document."”
Created: 6/13/2026, 12:23:11 AM
Your notes
Loading notes...