causal

Coarse-grained TPU layout limits vector-matrix bandwidth

The TPU's coarse-grained design allows larger systolic arrays that amortize register-file costs better, but it constrains data movement between vector and matrix units to just two lines of perimeter, whereas a GPU's many small units let data move through many more lines (e.g. 16); GPUs thus achieve higher vector-to-matrix bandwidth and cheaper intra-SM data movement, but cross-SM operation becomes complicated and expensive.

causalpending

Speaker

Reiner Pope

Evidence Quote

“The amount of data you can move between a vector unit and a matrix unit is actually much higher in a GPU than in a TPU.”

Source

Dwarkesh Patel Podcast - Reiner Pope – Chip design from the bottom up— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:33:43 AM

Your notes

Loading notes...