factual
Blackwell Chip-to-Chip Latency Limits Tensor Parallel Speedup
On Blackwell chips, point-to-point latency between chips can be about 4,000 nanoseconds, so an 8-way tensor parallel setup yields far less than an 8x improvement in tokens per second per user.
factualpending
Speaker
GavinEvidence Quote
“on Blackwell chips, it can be about 4,000 nanoseconds to go point to point.”
Source
Etched - Building AI Hardware to Make Inference Faster and Cheaper - [Invest Like the Best, EP.480]— Invest Like the BestCreated: 11/17/2026, 7:08:01 AM
Your notes
Loading notes...