factual

LLMs Trained Only to Predict Next Word

Although it feels like large language models plan whole documents, technically they are trained only to predict which word comes next; when trained on enough data this single objective produces fluent, coherent long-form text as if it had been planned.

factualpending

Speaker

Yejin Choi

Evidence Quote

“it's trained to predict which word comes next, but if you train it so well on so much data, then we realize that, "Wow, it can actually generate very nice fluent long document."”

Source

248 | Yejin Choi on AI and Common Sense— Sean Carroll's Mindscape
Created: 6/13/2026, 12:23:11 AM

Your notes

Loading notes...