DAgger algorithm
Dataset Aggregation: an imitation-learning method that collects expert corrections on the states a policy actually visits, relabeling them with better actions to make the policy robust to drift.
As defined in Dwarkesh Patel Podcast - Eric Jang – Building AlphaGo from scratch
Where it comes up · 1
Most important claims · 2 of 2Search all claims for “DAgger algorithm” →
- MCTS acts as a policy-improvement operator: for every action taken in a game (whether the game was won or lost), the search provides a strictly better action/distribution to imitate, giving one low-variance supervision target per move rather than a single noisy win/loss signal for the whole trajectory.Dwarkesh Patel Podcast - Eric Jang – Building AlphaGo from scratch
- Off-policy training in AlphaGo is fine—and even helpful in a DAgger sense—as long as the replay buffer is mostly states the current policy would visit plus a reasonable band of nearby states that funnel back to the optimal trajectory; it becomes harmful when MCTS relabels states the policy would never reach, since the network then wastes capacity learning good actions on unreachable states.Dwarkesh Patel Podcast - Eric Jang – Building AlphaGo from scratch