causal
Soft labels carry far more information than hard labels
Distillation is so effective per sample because soft targets (the full logit distribution) have far higher entropy than a one-hot token answer, carrying much more information per sample; AlphaGo exploits this by training the policy network to imitate the full MCTS visit-count distribution rather than just the single selected action ('dark knowledge' distillation).
causalpending
Speaker
Eric JangEvidence Quote
“In AlphaGo, you don't train the policy network to imitate the MCTS action. You train it to imitate the MCTS distribution.”
Created: 6/13/2026, 3:37:56 AM
Your notes
Loading notes...