H100
A high-end NVIDIA GPU used for training large AI models, drawing on the order of 1,000 watts.
As defined in et_995
Where it comes up · 22
- @Asianometry & Dylan Patel — How the semiconductor industry actually works4 claims
- Aravind Srinivas - Building An Answer Engine - [Invest Like the Best, EP.363]4 claims
- The AI Shortcut: How DeepSeek Cracked ChatGPT’s Secrets for Cheap4 claims
- Deepseek Dylan Patel Nathan Lambert3 claims
- The Geopolitics of AI Infrastructure - Dylan Patel, SemiAnalysis3 claims
- AI Market Jitters1 claim
- Capital, Compute & AI Scaling1 claim
- AGI is still 30 years away — Ege Erdil & Tamay Besiroglu1 claim
and 14 more
Most important claims · 10 of 35Search all claims for “H100” →
- Training requires a coherent cluster in which GPUs share a memory space so they effectively know what each other is computing; the largest coherent cluster until recently was 32,000 H100s, of which only 15,000-16,000 could run at once due to efficiency problems.Gavin Baker - AI, Semiconductors, and the Robotic Frontier - [Invest Like the Best, EP.385]
- The AI chip supply chain has a series of monopoly choke points—NVIDIA (chips), TSMC (fabrication, ~70% of worldwide chips), and ASML (lithography machines)—each holding roughly 70% market share, and these choke points can be restricted and monitored (as already done via US export controls denying China the H100's successor), making chip access the critical lever for coordinated slowdown or containment.332 - Can We Contain Artificial Intelligence
- AI mega-cluster power consumption has jumped from ~15-20 megawatts for GPT-4's 20,000 A100s in 2022 to ~150 megawatts for 100,000+ GPU clusters in 2024, a roughly 10x increase in two years, with US data center power consumption projected to reach ~10% of total US power by 2030 — and even that may be insufficient for the labs.Deepseek Dylan Patel Nathan Lambert
- DeepSeek's efficiency gains, contrary to causing NVIDIA's stock drop, illustrate Jevons paradox: as efficiency goes up, total resource consumption rises, evidenced by AWS H100 prices going up and H200s selling out after V3's launch because R1 wants the higher-memory chip.Deepseek Dylan Patel Nathan Lambert
- If small language models can be trained only on tokens important for reasoning (distilling intelligence from GPT-4) and made to bootstrap their own reasoning iteratively, you could decouple reasoning from memorized facts — a breakthrough that would remove the need for giant training clusters and disrupt the foundation-model players, since you'd no longer need a million-H100-equivalent cluster.Aravind Srinivas
- Google has the biggest cluster overall using TPUs, but spreads it across multiple data center super-regions ~30 miles apart connected by high-bandwidth fiber, whereas Elon's 200,000-GPU Memphis cluster is the largest fully connected single-site cluster; Google doesn't sell TPUs because they're optimized for internal workloads and selling would cannibalize its search profit DNA.Deepseek Dylan Patel Nathan Lambert
- Nvidia's CFO Colette disclosed on an earnings call that without export restrictions, Nvidia would have sold $50 billion worth of GPUs to China in the current year, and the export ban on the H20 (a cut-down H100 or H200 variant) prevented approximately one million GPUs from being sold.The Geopolitics of AI Infrastructure - Dylan Patel, SemiAnalysis
- A quantitative growth prediction: the human brain does roughly 1E15 flop/second, an H100 does computation on that order, an H100 costs ~$30K, and if it ran human-brain software earning human wages (~$50-100K/year), it would pay itself back on the order of a year — implying an economy doubling time of about a year, a specificity that qualitative Baumol objections fail to match.AGI is still 30 years away — Ege Erdil & Tamay Besiroglu
- Frontier compute is supply-constrained because GPUs are booked one or two years in advance, and a senior Meta researcher demanded 10,000 H100s, which represents billions of dollars over five to ten years, so even funded startups must wait while incumbents move to the next generation.Aravind Srinivas - Building An Answer Engine - [Invest Like the Best, EP.363]
- Due to US sanctions on China's advanced chips, top-tier GPUs like the H100 are no longer legally available to China, so Nvidia created a specialized chip for the Chinese market called the H800—a watered-down version with reduced clock speed but the same advanced architecture, performing at 60-77% of the H100's capability (1.3-1.6x slower).The AI Shortcut: How DeepSeek Cracked ChatGPT’s Secrets for Cheap