factual
Scale-out network is about 8x slower than scale-up
In Nvidia's rack topology, GPUs talk within a rack via fast NVLink (scale-up network) reaching any GPU in two hops through central NV switches, while leaving the rack uses a scale-out network up to a data-center switch that is typically about eight times slower in bandwidth.
factualpending
Speaker
Reiner PopeEvidence Quote
“The GPUs also have a much slower connectivity, which is typically about eight times slower... More generally, it’s called the scale-up network. You will typically also have a scale-out network”
Source
Dwarkesh Patel Podcast - Reiner Pope – The math behind how LLMs are trained and served— Dwarkesh Patel PodcastCreated: 6/13/2026, 3:34:09 AM
Your notes
Loading notes...