causal

AlphaGo's elegance: never starting at zero success

AlphaGo is an elegant RL algorithm because it never has to start at a zero-percent success rate or solve the exploration problem of reaching non-zero success; every step is stable supervised learning (value classification plus policy KL minimization) on improved labels, so you can train arbitrarily large networks stably with simple infrastructure and never sit in a flat-gradient local minimum.

causalpending

Speaker

Eric Jang

Evidence Quote

“there's actually no TD error learning or dynamic programming, at least explicitly. It's just supervised learning on a value classification as well as a policy KL minimization. It's just a supervised learning problem on improved labels.”

Source

Dwarkesh Patel Podcast - Eric Jang – Building AlphaGo from scratch— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:37:56 AM

Your notes

Loading notes...