causal

Hallucination From Exponential Error Accumulation

LLM hallucination arises because in autoregressive prediction each token has a non-zero probability of taking the output outside the set of correct answers, and assuming these errors are roughly independent, the probability of remaining correct decreases exponentially with the number of tokens produced.

causalpending

Speaker

Yann LeCun

Evidence Quote

“the probability that an answer would be nonsensical increases exponentially with the number of tokens.”

Source

Research Pioneer— Lex Fridman Podcast
Created: 6/13/2026, 3:30:53 AM

Your notes

Loading notes...