factual
Frontier inference is mostly single scale-up domain
Frontier labs doing inference are largely within a single scale-up domain: they maximize expert parallelism up to the scale-up domain size and do very little or no pipelining (DeepSeek reports doing a lot of expert parallelism), reserving some pipelining only for models too large to fit a rack's memory; tensor parallelism is no longer profitable because experts are now so small.
factualpending
Speaker
Reiner PopeEvidence Quote
“you should increase your expert parallelism up to your scale-up domain size, and then do very little pipelining. Maybe none at all, maybe two”
Source
Dwarkesh Patel Podcast - Reiner Pope – The math behind how LLMs are trained and served— Dwarkesh Patel PodcastCreated: 6/13/2026, 3:34:09 AM
Your notes
Loading notes...