SM
Streaming Multiprocessor; the core compute units on a GPU (a bit over a hundred per GPU) that DeepSeek scheduled manually to manage communications.
As defined in Deepseek Dylan Patel Nathan Lambert
Where it comes up · 2
- Dwarkesh Patel Podcast - Reiner Pope – Chip design from the bottom up3 claims
- Deepseek Dylan Patel Nathan Lambert1 claim
Most important claims · 4 of 4Search all claims for “SM” →
- At the top level a GPU is a regular grid of near-identical SMs around an L2 memory, while a TPU is a few coarse-grained matrix units (large systolic arrays) with a vector unit in the middle; since an SM is essentially a scaled-down matrix unit plus small vector unit, a GPU can be understood as a lot of tiny TPUs tiled across the whole chip.Dwarkesh Patel Podcast - Reiner Pope – Chip design from the bottom up
- The TPU's coarse-grained design allows larger systolic arrays that amortize register-file costs better, but it constrains data movement between vector and matrix units to just two lines of perimeter, whereas a GPU's many small units let data move through many more lines (e.g. 16); GPUs thus achieve higher vector-to-matrix bandwidth and cheaper intra-SM data movement, but cross-SM operation becomes complicated and expensive.Dwarkesh Patel Podcast - Reiner Pope – Chip design from the bottom up
- Because the H800 GPUs shipped to China legally had cut interconnect bandwidth, DeepSeek bypassed NVIDIA's standard NCCL communications library and manually scheduled communications by assigning specific SMs (GPU cores) to tasks using PTX, an assembly-level approach most companies avoid, in order to fully utilize their constrained hardware.Deepseek Dylan Patel Nathan Lambert
- MatX has publicly discussed a 'splittable systolic array' — big systolic arrays that can also act as small systolic arrays — as a way to capture the GPU-like benefit of smaller flexible structures surrounded by SRAM while discarding the space-consuming CUDA-architecture support hardware that GPUs require in each SM.Dwarkesh Patel Podcast - Reiner Pope – Chip design from the bottom up