Sheela
Sheela
Home
Add
Questions
Thinkers
Claims
People
Concepts
Jargon
Sources
Health
Predictions
Collections
Domain Landscape
Contribute
Review
Dedup
Topic requests
Settings
Help

← Jargon

Jargonmachine learning· 3 claims· 1 episode

autoregressive LLM

A large language model constrained to predict the next token using only the tokens before it, generating text one token at a time by sampling from a probability distribution over the vocabulary.

As defined in Research Pioneer


Where it comes up · 1
  • Research Pioneer3 claims

Most important claims · 3 of 3Search all claims for “autoregressive LLM” →
  • Autoregressive LLMs cannot achieve human-level intelligence because they lack or only primitively possess four essential characteristics of intelligent behavior: understanding the physical world, persistent memory, reasoning, and planning.Research Pioneer
  • Building world models by training generative models to predict future video frames fails because you cannot represent probability distributions over high-dimensional continuous spaces (all possible frames) the way you can over a finite dictionary of text tokens; the world is too rich in unpredictable detail.Research Pioneer
  • LLM hallucination arises because in autoregressive prediction each token has a non-zero probability of taking the output outside the set of correct answers, and assuming these errors are roughly independent, the probability of remaining correct decreases exponentially with the number of tokens produced.Research Pioneer