causal

Bigger scale-up cuts memory time via parallel weight loads

Larger scale-up domains massively reduce weight memory-fetch latency because all GPUs in a scale-up domain load weights in parallel, so effective bandwidth equals scale-up size times per-GPU bandwidth; this term grew ~8x from Hopper while per-GPU bandwidth only grows ~1.5-2x per generation, meaning scale-up size—not memory capacity—solves the bandwidth/latency problem.

causalpending

Speaker

Reiner Pope

Evidence Quote

“I can use all the GPUs in my scale-up domain in parallel to load the weights... this memory bandwidth term itself is equal to scale-up size... this one increased by a factor of 8 from Hopper.”

Source

Dwarkesh Patel Podcast - Reiner Pope – The math behind how LLMs are trained and served— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:34:09 AM

Your notes

Loading notes...