Claude Shannon's information theory framed language as a sequence of predictions where each word is chosen from a set of likely words given what came before; building on this, 1980s researchers trained tiny networks to predict the next word, and these networks spontaneously clustered similar words (verbs with verbs, nouns with nouns, words of similar meaning).

factualpending

Speaker

Narrator

Evidence Quote

“each word you say is chosen from a set of likely words given what came before”

Source

How LLMs Took Over The World— Art of the Problem
Created: 6/18/2026, 2:18:12 PM

Your notes

Loading notes...