factual

Shared-nothing GPU fabric shows sublinear sharding returns

Sharding a large matrix multiply across up to eight GPUs with NVLink reduces latency but scales sublinearly, so eight times the hardware yields perhaps 4 to 5 times the speed due to communication overhead, making tensor-parallel sharding a poor choice for throughput-optimized serving.

factualpending

Speaker

Neil Movva

Evidence Quote

“It's sublinear scaling. You'll use eight times more hardware, but you won't get eight times the speed.”

Source

Neil Movva - Making AI 10x Cheaper - [Invest Like the Best, EP.488]— Invest Like the Best
Created: 11/17/2026, 6:43:49 AM

Your notes

Loading notes...