factual

GPT-3 finds best-matching Markov chain

Language generation systems like GPT-3 are in some sense trying to find the Markov chain that best matches the language production they can observe, drawing on a gigantic corpus of internet English text, though worked hard to add more long-range memory than a basic Markov chain.

factualpending

Speaker

Jordan Ellenberg

Evidence Quote

“trying to find the Markov chain that best matches the language production it is able to see”

Source

151 | Jordan Ellenberg on the Mathematics of Political Boundaries— Sean Carroll's Mindscape
Created: 6/13/2026, 12:30:48 AM

Your notes

Loading notes...