factual
Q-learning plans over visited trajectories, MCTS over unvisited
There is a deep duality between MCTS and Q-learning: both propagate value information through a recursive dynamic-programming structure, but MCTS plans over trajectories the agent hasn't visited yet (forward search), while Q-learning plans over trajectories it has already visited—Q-learning mattered historically because we couldn't do forward search on high-dimensional problems like robotics.
factualpending
Speaker
Eric JangEvidence Quote
“Q-learning, or approximate dynamic programming, propagates what you know about the future Qs backward like this. You can see that there's a similar structure going on here”
Created: 6/13/2026, 3:37:56 AM
Your notes
Loading notes...