Autoregressive token prediction in LLMs produces exponential divergence errors because each generated token carries independent error probability, and if any token falls outside the set of reasonable answers, there is no way to recover afterward, leading to hallucinations

causalpending

Speaker

Yann LeCun

Evidence Quote

“every time you produce a token there is some chance that the token is not within the set of reasonable answers and take you outside a set of reasonable answers and if it does that there is no way to fix it afterwards”

Source

The Shape of AI to Come! Yann LeCun at AI Action Summit 2025— DSAI by Dr. Osbert Tay
Created: 8/11/2026, 7:07:43 AM

Your notes

Loading notes...