Sheela
Sheela
Home
Add
Questions
Thinkers
Claims
People
Concepts
Jargon
Sources
Health
Predictions
Collections
Domain Landscape
Contribute
Review
Dedup
Topic requests
Settings
Help

← Jargon

Jargontechnology· 2 claims· 1 episode

vector unit

A processing unit performing element-wise vector operations, sitting between matrix units on a TPU and present in GPUs.

As defined in Dwarkesh Patel Podcast - Reiner Pope – Chip design from the bottom up


Where it comes up · 1
  • Dwarkesh Patel Podcast - Reiner Pope – Chip design from the bottom up2 claims

Most important claims · 2 of 2Search all claims for “vector unit” →
  • At the top level a GPU is a regular grid of near-identical SMs around an L2 memory, while a TPU is a few coarse-grained matrix units (large systolic arrays) with a vector unit in the middle; since an SM is essentially a scaled-down matrix unit plus small vector unit, a GPU can be understood as a lot of tiny TPUs tiled across the whole chip.Dwarkesh Patel Podcast - Reiner Pope – Chip design from the bottom up
  • The TPU's coarse-grained design allows larger systolic arrays that amortize register-file costs better, but it constrains data movement between vector and matrix units to just two lines of perimeter, whereas a GPU's many small units let data move through many more lines (e.g. 16); GPUs thus achieve higher vector-to-matrix bandwidth and cheaper intra-SM data movement, but cross-SM operation becomes complicated and expensive.Dwarkesh Patel Podcast - Reiner Pope – Chip design from the bottom up