factual
Self-supervised next-word prediction is AI-complete
Language modeling—producing a probability distribution over the next word—is an ideal problem because it is simple to state, has essentially infinite self-supervised training data from the text of the web, and is AI-complete: doing it well means you can do essentially anything.
factualpending
Speaker
Noam ShazeerEvidence Quote
“it is very, very simple to state: give me a probability distribution over the next word. Also, there's roughly infinite training data out there... And then it's AI complete.”
Created: 6/13/2026, 3:30:41 AM
Your notes
Loading notes...