causal

RL is less compute-efficient than pre-training

RL training is less efficient per token than pre-training for two reasons: you may not train a backward pass on all rollouts (so FLOPs per token can be as low as 2 rather than the full forward+backward 6), and RL involves substantial decode which often runs at lower MFU than training, together giving roughly a 30% efficiency penalty.

causalpending

Speaker

Reiner Pope

Evidence Quote

“the other, perhaps even bigger inefficiency is that this involves a substantial amount of decode. Often decode runs at less MFU than training.”

Source

Dwarkesh Patel Podcast - Reiner Pope – The math behind how LLMs are trained and served— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:34:09 AM

Your notes

Loading notes...