causal

Distillation amortizes search into the policy network

By distilling the MCTS-improved distribution back into the raw policy network, you amortize the work that thousands of search simulations do into a single forward pass, so the next round of self-play starts from a much stronger base and reaches higher win rates for the same simulation budget.

causalpending

Speaker

Eric Jang

Evidence Quote

“over time the raw network actually takes all of the burden of that big TPU pod and just pushes it into the network. You can do all of that work with one neural network forward pass.”

Source

Dwarkesh Patel Podcast - Eric Jang – Building AlphaGo from scratch— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:37:56 AM

Your notes

Loading notes...