Tokens are discrete representations from a finite set (typically ~100,000 tokens in LLMs), but the physical world is continuous and high-dimensional, making token-based prediction unsuitable for modeling natural continuous phenomena.

factualpending

Speaker

Yann LeCun

Evidence Quote

“Tokens are discreet... in a typical LLM the number of possible tokens is on the order of 100 thousand or something like that right... natural data that is highdimensional and continuous and every attempt at trying to get system to understand the world or build mental models of the world by being trained to predict videos at the pixel level basically have failed”

Source

Metas AI Boss Says He DONE With LLMS...— TheAIGRID
Created: 8/11/2026, 7:13:18 AM

Your notes

Loading notes...