Tokens are discrete representations from a finite set (typically ~100,000 tokens in LLMs), but the physical world is continuous and high-dimensional, making token-based prediction unsuitable for modeling natural continuous phenomena.
factualpending
Speaker
Yann LeCunEvidence Quote
“Tokens are discreet... in a typical LLM the number of possible tokens is on the order of 100 thousand or something like that right... natural data that is highdimensional and continuous and every attempt at trying to get system to understand the world or build mental models of the world by being trained to predict videos at the pixel level basically have failed”
Created: 8/11/2026, 7:13:18 AM
Your notes
Loading notes...