NVL72
Nvidia's rack-scale system ganging 72 GPUs together into one tightly-coupled compute unit.
As defined in Dwarkesh Patel Podcast - Jensen Huang – TPU competition, why we should sell chips to China, & Nvidia’s supply chain moat
Where it comes up · 4
- Etched - Building AI Hardware to Make Inference Faster and Cheaper - [Invest Like the Best, EP.480]2 claims
- Inside the Trillion-Dollar AI Buildout1 claim
- Dwarkesh Patel Podcast - Jensen Huang – TPU competition, why we should sell chips to China, & Nvidia’s supply chain moat1 claim
- Neil Movva - Making AI 10x Cheaper - [Invest Like the Best, EP.488]1 claim
Most important claims · 5 of 5Search all claims for “NVL72” →
- Serving giant models to billions of users requires many chips to communicate quickly with each other, because fast intra-cluster communication alone is insufficient at tens of thousands of chips, and infrastructure must be built across the full stack from wafer to watt.Etched - Building AI Hardware to Make Inference Faster and Cheaper - [Invest Like the Best, EP.480]
- Scale-up domains that were 8 chips and then 72 with NVL72 could grow far larger, potentially into $40 to $100 billion mega token factories serving one or a few models to massive user bases, following the economies-of-scale logic of single monolithic fabs.Etched - Building AI Hardware to Make Inference Faster and Cheaper - [Invest Like the Best, EP.480]
- The most promising hardware innovation area is networking between chips, including electrical-to-optical bridging, because extending memory across chips is easier than inventing new memory technology, which DRAM's capital base makes hard to attack.Inside the Trillion-Dollar AI Buildout
- Export controls cannot deny China meaningful AI compute, because China has abundant energy, manufactures the majority of mainstream chips, holds roughly 50% of the world's AI researchers, and can simply gang together more 7nm chips (essentially Hopper-class) since energy is effectively free for them—so they have already passed any compute threshold relevant to cyber-offensive concerns.Dwarkesh Patel Podcast - Jensen Huang – TPU competition, why we should sell chips to China, & Nvidia’s supply chain moat
- The unit of optimization is now the whole rack or cluster rather than one GPU, since Nvidia ships systems like NVL72 as prescribed racks of 72 units, and whoever programs rack-scale compute most efficiently will win on latency and throughput.Neil Movva - Making AI 10x Cheaper - [Invest Like the Best, EP.488]