causal
20ms train interval from HBM drain time
The ~20 millisecond batch interval comes from wanting to read all of HBM capacity into the chip exactly once per forward pass—capacity divided by bandwidth—which is about 20 milliseconds across many HBM generations (e.g., Rubin's ~288 GB over ~20 TB/s ≈ 15 ms); running slower would wastefully re-read read-only weights and KVs.
causalpending
Speaker
Reiner PopeEvidence Quote
“we want to read all of the memory capacity into the chip. That is capacity divided by bandwidth. That tends to be 20 milliseconds on many different generations of HBM.”
Source
Dwarkesh Patel Podcast - Reiner Pope – The math behind how LLMs are trained and served— Dwarkesh Patel PodcastCreated: 6/13/2026, 3:34:09 AM
Your notes
Loading notes...