Tensor core
Nvidia's hardware unit for accelerating matrix multiplication, a specific implementation of a systolic array, introduced in the Volta generation.
As defined in Dwarkesh Patel Podcast - Reiner Pope – Chip design from the bottom up
Other definitions
Specialized hardware units in Nvidia GPUs designed specifically for matrix operations and tensor computations that are fundamental to deep learning, providing orders of magnitude speedup for neural network training and inference.
Ep17. Welcome Jensen Huang | BG2 w/ Bill Gurley & Brad GerstnerA specialized hardware unit within a GPU designed to perform high-throughput matrix multiplication operations, essential for training and running neural networks and large language models.
How AI Datacenters Eat the World
Where it comes up · 3
- Dwarkesh Patel Podcast - Reiner Pope – Chip design from the bottom up2 claims
- Neil Movva - Making AI 10x Cheaper - [Invest Like the Best, EP.488]2 claims
- Ep17. Welcome Jensen Huang | BG2 w/ Bill Gurley & Brad Gerstner1 claim
Most important claims · 5 of 5Search all claims for “Tensor core” →
- The first tensor cores occupied only about 5 to 10 percent of die area after fierce internal competition with graphics teams, because silicon allocation is guarded and Nvidia had to bet early on deep learning demand.Neil Movva - Making AI 10x Cheaper - [Invest Like the Best, EP.488]
- At the top level a GPU is a regular grid of near-identical SMs around an L2 memory, while a TPU is a few coarse-grained matrix units (large systolic arrays) with a vector unit in the middle; since an SM is essentially a scaled-down matrix unit plus small vector unit, a GPU can be understood as a lot of tiny TPUs tiled across the whole chip.Dwarkesh Patel Podcast - Reiner Pope – Chip design from the bottom up
- A systolic array bakes two levels of the matrix-multiply loop into hardware so that the weight matrix is stored locally in registers and reused across many input vectors; this provides x×y as much compute while keeping input/output communication at only x, solving the data-movement problem that plagued CUDA cores and making it the most efficient known circuit for matrix multiplication.Dwarkesh Patel Podcast - Reiner Pope – Chip design from the bottom up
- Nvidia achieved a marginal cost reduction in computing of 100,000x over the course of 10 years, with Moore's Law alone accounting for about 100x, through accelerated computing, tensor cores, HBM, NVLink, new numerical precisions, and new architectures.Ep17. Welcome Jensen Huang | BG2 w/ Bill Gurley & Brad Gerstner
- Tensor cores and matrix multiplication are the atomic unit of computation because linear algebra compactly represents arbitrary relationships in data, though Neil admits there is no deeper proof for why matrix multiplies are so central.Neil Movva - Making AI 10x Cheaper - [Invest Like the Best, EP.488]