factual

Scale-out network is about 8x slower than scale-up

In Nvidia's rack topology, GPUs talk within a rack via fast NVLink (scale-up network) reaching any GPU in two hops through central NV switches, while leaving the rack uses a scale-out network up to a data-center switch that is typically about eight times slower in bandwidth.

factualpending

Speaker

Reiner Pope

Evidence Quote

“The GPUs also have a much slower connectivity, which is typically about eight times slower... More generally, it’s called the scale-up network. You will typically also have a scale-out network”

Source

Dwarkesh Patel Podcast - Reiner Pope – The math behind how LLMs are trained and served— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:34:09 AM

Your notes

Loading notes...