factual
Blackwell racks have surplus memory for trillion-param models
A Blackwell rack has many tens of terabytes of memory, far exceeding the ~1 terabyte needed for a trillion-parameter model, so even in inference pipelining is rarely used because it reduces a memory-capacity number that is already comfortably small—implying hardware could have been designed with less HBM per GPU.
factualpending
Speaker
Reiner PopeEvidence Quote
“a rack of Blackwell has many tens of terabytes. That’s much bigger than a trillion parameter model. A trillion parameter model only needs one terabyte, so it already fits.”
Source
Dwarkesh Patel Podcast - Reiner Pope – The math behind how LLMs are trained and served— Dwarkesh Patel PodcastCreated: 6/13/2026, 3:34:09 AM
Your notes
Loading notes...