factual
Smaller batch best for convergence, worse for systems
There is a fundamental tension in training batch size: from an ML convergence-rate perspective smaller is always better because the gradient descent step uses the freshest information, while from a systems perspective smaller is worse; the optimal batch size is the trade-off between these two forces.
factualpending
Speaker
Reiner PopeEvidence Quote
“From an ML convergence rate perspective, smaller is always better because you’re getting the freshest information from the gradient descent.”
Source
Dwarkesh Patel Podcast - Reiner Pope – The math behind how LLMs are trained and served— Dwarkesh Patel PodcastCreated: 6/13/2026, 3:34:09 AM
Your notes
Loading notes...