factual
Coherent Clusters Require Shared Memory Across GPUs
Training requires a coherent cluster in which GPUs share a memory space so they effectively know what each other is computing; the largest coherent cluster until recently was 32,000 H100s, of which only 15,000-16,000 could run at once due to efficiency problems.
factualpending
Speaker
Gavin BakerEvidence Quote
“So you had 32,000 H100s probably only using like, at most, 15,000, 16,000 of those H100s at once because of efficiency problems.”
Source
Gavin Baker - AI, Semiconductors, and the Robotic Frontier - [Invest Like the Best, EP.385]— Invest Like the BestCreated: 11/12/2026, 2:50:15 AM
Your notes
Loading notes...