causal
RL is less compute-efficient than pre-training
RL training is less efficient per token than pre-training for two reasons: you may not train a backward pass on all rollouts (so FLOPs per token can be as low as 2 rather than the full forward+backward 6), and RL involves substantial decode which often runs at lower MFU than training, together giving roughly a 30% efficiency penalty.
causalpending
Speaker
Reiner PopeEvidence Quote
“the other, perhaps even bigger inefficiency is that this involves a substantial amount of decode. Often decode runs at less MFU than training.”
Source
Dwarkesh Patel Podcast - Reiner Pope – The math behind how LLMs are trained and served— Dwarkesh Patel PodcastCreated: 6/13/2026, 3:34:09 AM
Your notes
Loading notes...