causal

PUCT exploration is unsuited to LLM action spaces

MCTS works for Go because value estimation is concrete (groundable) and the action space is discrete and bounded, suiting the PUCT exploration heuristic; for LLMs this breaks down because the √N/(1+Na) term assumes you re-sample the same child, but language's vast open-ended action space means you almost never sample the same continuation twice, so PUCT is too greedy and not the right way to guide tree search.

causalpending

Speaker

Eric Jang

Evidence Quote

“because language is so broad and open-ended, a discrete set of actions is not really an appropriate choice for an LLM... this type of exploration heuristic is probably not the right thing to guide how to search down a tree.”

Source

Dwarkesh Patel Podcast - Eric Jang – Building AlphaGo from scratch— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:37:56 AM

Your notes

Loading notes...