AlphaGo was essentially a scaling-up of Gerry Tesauro's TD-Gammon (which used temporal difference learning to beat the world's best backgammon players), plus an additional search innovation; AlphaGo itself waited for final outcomes rather than using TD, but AlphaZero used TD and applied it across many games extremely well.

factualpending

Speaker

Richard Sutton

Evidence Quote

“AlphaGo was merely a scaling up of that process... AlphaGo actually didn't use TD learning. It waited to see the final outcomes. But AlphaZero used TD.”

Source

Richard Sutton – Father of RL thinks LLMs are a dead end— Dwarkesh Patel
Created: 6/18/2026, 1:59:29 PM

Your notes

Loading notes...