factual

Blackwell racks have surplus memory for trillion-param models

A Blackwell rack has many tens of terabytes of memory, far exceeding the ~1 terabyte needed for a trillion-parameter model, so even in inference pipelining is rarely used because it reduces a memory-capacity number that is already comfortably small—implying hardware could have been designed with less HBM per GPU.

factualpending

Speaker

Reiner Pope

Evidence Quote

“a rack of Blackwell has many tens of terabytes. That’s much bigger than a trillion parameter model. A trillion parameter model only needs one terabyte, so it already fits.”

Source

Dwarkesh Patel Podcast - Reiner Pope – The math behind how LLMs are trained and served— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:34:09 AM

Your notes

Loading notes...