Autoregressive token prediction in LLMs produces exponential divergence errors because each generated token carries independent error probability, and if any token falls outside the set of reasonable answers, there is no way to recover afterward, leading to hallucinations
causalpending
Speaker
Yann LeCunEvidence Quote
“every time you produce a token there is some chance that the token is not within the set of reasonable answers and take you outside a set of reasonable answers and if it does that there is no way to fix it afterwards”
Created: 8/11/2026, 7:07:43 AM
Your notes
Loading notes...