causal

Advantage estimation isolates the moves that mattered

The optimal RL target discards the moves that did not contribute and gradients only on the one move that actually made you better; advantage estimation (pushing up actions better than average, down those worse than average) approximates this, requiring a good estimate of average performance from a state—bringing back the value function.

causalpending

Speaker

Eric Jang

Evidence Quote

“Ideally, what you really want to do in RL is push up the actions that make you better than average and push down the actions that make you worse than average. They call this advantage.”

Source

Dwarkesh Patel Podcast - Eric Jang – Building AlphaGo from scratch— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:37:56 AM

Your notes

Loading notes...