causal

MCTS lets you improve locally without solving the problem

The reason MCTS is so much more sample-efficient than LLM/robotics RL is that you can locally improve your policy per move—the search, truncated by a working value function, suggests a better next move even without unrolling the whole trajectory or being able to win the game—whereas in normal LLM/robotics RL there is no way to evaluate and improve a move independent of actually solving the whole problem.

causalpending

Speaker

Dwarkesh Patel

Evidence Quote

“in Go, you know that MCTS is almost certainly better than your current policy, even though you haven't explored the end of any trajectory. In normal reasoning for LLMs or robotics, there's no way to just locally evaluate and improve your next move in a way that's independent of actually solving the problem.”

Source

Dwarkesh Patel Podcast - Eric Jang – Building AlphaGo from scratch— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:37:56 AM

Your notes

Loading notes...