factual

Self-supervised next-word prediction is AI-complete

Language modeling—producing a probability distribution over the next word—is an ideal problem because it is simple to state, has essentially infinite self-supervised training data from the text of the web, and is AI-complete: doing it well means you can do essentially anything.

factualpending

Speaker

Noam Shazeer

Evidence Quote

“it is very, very simple to state: give me a probability distribution over the next word. Also, there's roughly infinite training data out there... And then it's AI complete.”

Source

Jeff Dean & Noam Shazeer — 25 years at Google: from PageRank to AGI— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:30:41 AM

Your notes

Loading notes...