autoregressive generation
Process of generating output one token at a time by predicting the next token based on all previous tokens, then feeding the predicted token back into the input for the next prediction
As defined in The Shape of AI to Come! Yann LeCun at AI Action Summit 2025
Other definitions
Generation process that produces sequences one element at a time, conditioning each new element on all previous elements, then feeding the generated element back as input for the next step, used by most current language models
The Shape of AI to Come! Yann LeCun at AI Action Summit 2025A generation approach where a model produces outputs sequentially, one element at a time, feeding the previously generated element back into the model as input to generate the next element
Yann LeCun's $1B Bet Against LLMs [Part 1]
Where it comes up · 2
- 363 | Chandra Sripada on How LLMs and Humans are Cognitive Cousins2 claims
- The Shape of AI to Come! Yann LeCun at AI Action Summit 20251 claim
Most important claims · 3 of 3Search all claims for “autoregressive generation” →
- Next-word prediction as a training objective and generation method is compatible with planning: the Anthropic rhyme example shows a model generating the final word 'rabbit' before earlier words, revealed by mechanistic interpretability, so autoregressive training can encapsulate staged problem-solving.363 | Chandra Sripada on How LLMs and Humans are Cognitive Cousins
- Next-word prediction on internet-scale data is the most powerful training signal, because it produces a dense and high-quality representational infrastructure, and many cognitive principles in both LLMs and humans are downstream of prediction.363 | Chandra Sripada on How LLMs and Humans are Cognitive Cousins
- Causal architectures in autoregressive generation prevent cheating (using future tokens to predict past ones) by restricting the transformer to only look at previous tokens in the sequence.The Shape of AI to Come! Yann LeCun at AI Action Summit 2025