factual

Frontier inference is mostly single scale-up domain

Frontier labs doing inference are largely within a single scale-up domain: they maximize expert parallelism up to the scale-up domain size and do very little or no pipelining (DeepSeek reports doing a lot of expert parallelism), reserving some pipelining only for models too large to fit a rack's memory; tensor parallelism is no longer profitable because experts are now so small.

factualpending

Speaker

Reiner Pope

Evidence Quote

“you should increase your expert parallelism up to your scale-up domain size, and then do very little pipelining. Maybe none at all, maybe two”

Source

Dwarkesh Patel Podcast - Reiner Pope – The math behind how LLMs are trained and served— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:34:09 AM

Your notes

Loading notes...