factual

Blackwell Chip-to-Chip Latency Limits Tensor Parallel Speedup

On Blackwell chips, point-to-point latency between chips can be about 4,000 nanoseconds, so an 8-way tensor parallel setup yields far less than an 8x improvement in tokens per second per user.

factualpending

Speaker

Gavin

Evidence Quote

“on Blackwell chips, it can be about 4,000 nanoseconds to go point to point.”

Source

Etched - Building AI Hardware to Make Inference Faster and Cheaper - [Invest Like the Best, EP.480]— Invest Like the Best
Created: 11/17/2026, 7:08:01 AM

Your notes

Loading notes...