factual

LLMs Learning Meaning From Next-Token Prediction

There is evidence that large language models can acquire a form of meaning despite being trained only to predict the next token, which connects to emergence because higher-level semantic competence appears as a collective property not explicitly programmed, illustrating how complex capabilities can emerge from simple objective functions.

factualpending

Speaker

Sean Carroll

Evidence Quote

“What do you see as the connection between LLMs and emergence? What interests you about it, if anything?”

Source

AMA | June 2023— Sean Carroll's Mindscape
Created: 6/13/2026, 12:18:30 AM

Your notes

Loading notes...