causal
MCTS lets you improve locally without solving the problem
The reason MCTS is so much more sample-efficient than LLM/robotics RL is that you can locally improve your policy per move—the search, truncated by a working value function, suggests a better next move even without unrolling the whole trajectory or being able to win the game—whereas in normal LLM/robotics RL there is no way to evaluate and improve a move independent of actually solving the whole problem.
causalpending
Speaker
Dwarkesh PatelEvidence Quote
“in Go, you know that MCTS is almost certainly better than your current policy, even though you haven't explored the end of any trajectory. In normal reasoning for LLMs or robotics, there's no way to just locally evaluate and improve your next move in a way that's independent of actually solving the problem.”
Created: 6/13/2026, 3:37:56 AM
Your notes
Loading notes...