Claude Shannon's information theory framed language as a sequence of predictions where each word is chosen from a set of likely words given what came before; building on this, 1980s researchers trained tiny networks to predict the next word, and these networks spontaneously clustered similar words (verbs with verbs, nouns with nouns, words of similar meaning).
factualpending
Speaker
NarratorEvidence Quote
“each word you say is chosen from a set of likely words given what came before”
Created: 6/18/2026, 2:18:12 PM
Your notes
Loading notes...