Current LLMs operate by autoregressive token prediction where a causal transformer repeatedly takes a window of previous tokens and predicts the next token, and this creates a fundamental problem: each token prediction has some probability of being wrong, and if one is, there is no way to fix it afterwards, leading to exponential divergence and hallucination issues.

causalpending

Speaker

Yann LeCun

Evidence Quote

“every time you produce a token there is some chance that the token is not within the set of reasonable answers and if it does that there is no way to fix it afterwards...you get exponential Divergence”

Source

The Shape of AI to Come! Yann LeCun at AI Action Summit 2025— DSAI by Dr. Osbert Tay
Created: 8/11/2026, 1:17:36 AM

Your notes

Loading notes...