causal
Advantage estimation isolates the moves that mattered
The optimal RL target discards the moves that did not contribute and gradients only on the one move that actually made you better; advantage estimation (pushing up actions better than average, down those worse than average) approximates this, requiring a good estimate of average performance from a state—bringing back the value function.
causalpending
Speaker
Eric JangEvidence Quote
“Ideally, what you really want to do in RL is push up the actions that make you better than average and push down the actions that make you worse than average. They call this advantage.”
Created: 6/13/2026, 3:37:56 AM
Your notes
Loading notes...