factual

Smaller batch best for convergence, worse for systems

There is a fundamental tension in training batch size: from an ML convergence-rate perspective smaller is always better because the gradient descent step uses the freshest information, while from a systems perspective smaller is worse; the optimal batch size is the trade-off between these two forces.

factualpending

Speaker

Reiner Pope

Evidence Quote

“From an ML convergence rate perspective, smaller is always better because you’re getting the freshest information from the gradient descent.”

Source

Dwarkesh Patel Podcast - Reiner Pope – The math behind how LLMs are trained and served— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:34:09 AM

Your notes

Loading notes...