factual
Shared-nothing GPU fabric shows sublinear sharding returns
Sharding a large matrix multiply across up to eight GPUs with NVLink reduces latency but scales sublinearly, so eight times the hardware yields perhaps 4 to 5 times the speed due to communication overhead, making tensor-parallel sharding a poor choice for throughput-optimized serving.
factualpending
Speaker
Neil MovvaEvidence Quote
“It's sublinear scaling. You'll use eight times more hardware, but you won't get eight times the speed.”
Created: 11/17/2026, 6:43:49 AM
Your notes
Loading notes...