factual

Coherent Clusters Require Shared Memory Across GPUs

Training requires a coherent cluster in which GPUs share a memory space so they effectively know what each other is computing; the largest coherent cluster until recently was 32,000 H100s, of which only 15,000-16,000 could run at once due to efficiency problems.

factualpending

Speaker

Gavin Baker

Evidence Quote

“So you had 32,000 H100s probably only using like, at most, 15,000, 16,000 of those H100s at once because of efficiency problems.”

Source

Gavin Baker - AI, Semiconductors, and the Robotic Frontier - [Invest Like the Best, EP.385]— Invest Like the Best
Created: 11/12/2026, 2:50:15 AM

Your notes

Loading notes...