TCO
Total Cost of Ownership; the full multi-year cost of a cluster including GPUs, data center, power, debt, and operations.
As defined in @Asianometry & Dylan Patel — How the semiconductor industry actually works
Where it comes up · 5
- Dwarkesh Patel Podcast - Dylan Patel — Deep dive on the 3 big bottlenecks to scaling AI compute3 claims
- Dwarkesh Patel Podcast - Jensen Huang – TPU competition, why we should sell chips to China, & Nvidia’s supply chain moat2 claims
- Satya Nadella — How Microsoft is preparing for AGI2 claims
- Nvidia v. Google, Scaling Laws, and the Economics of AI1 claim
- @Asianometry & Dylan Patel — How the semiconductor industry actually works1 claim
Most important claims · 9 of 9Search all claims for “TCO” →
- When power is the binding constraint, the price of compute becomes irrelevant, so the most advanced chips and data centers that produce more tokens per watt will win on pricing power even if they cost much more.Nvidia v. Google, Scaling Laws, and the Economics of AI
- Because semiconductors and deployment are so supply-limited, a GPU is not priced by the comparative new chip you could buy but by the value of work it can do today; since newer models like GPT-5.4 are cheaper to serve, higher quality, and have a far larger TAM than GPT-4, an H100 produces more valuable tokens and is therefore worth more today than three years ago.Dwarkesh Patel Podcast - Dylan Patel — Deep dive on the 3 big bottlenecks to scaling AI compute
- In a large GPU cluster's total cost of ownership (multi-year, including debt and operations), roughly 80% is the GPUs/servers, ~10% the data center, and ~10% (under 15%, actually under 10%) the power - so the marginal cost of power is nearly irrelevant; the real constraint is the ability to generate and deliver new power, not its price, which is why clusters won't run only in daytime even with cheap solar.@Asianometry & Dylan Patel — How the semiconductor industry actually works
- Power will not be the binding US constraint because beyond the three combined-cycle gas turbine makers there are 16+ gas power-gen vendors plus aeroderivatives, medium-speed reciprocating engines, ship engines, fuel cells (Bloom), and solar-plus-battery; even doubling power cost to ~$3,500/kW only raises a GPU's TCO a few cents/hour, and utility-scale batteries could unlock ~20% of the terawatt-scale US grid that sits idle outside peak hours.Dwarkesh Patel Podcast - Dylan Patel — Deep dive on the 3 big bottlenecks to scaling AI compute
- H100 rental prices have inflected upward, with some AI labs signing two-to-three-year deals as high as $2.40/hour for Hopper even though it costs ~$1.40/hour to build over five years—margins far above the original ~35%—because rolling-off short-term contracts let willing buyers crowd out other suppliers.Dwarkesh Patel Podcast - Dylan Patel — Deep dive on the 3 big bottlenecks to scaling AI compute
- Microsoft has full access to OpenAI's silicon/chip program IP (the only excluded IP being consumer hardware), partly because Microsoft gave OpenAI IP to bootstrap them and built supercomputers together, so as OpenAI innovates at the system level Microsoft gets access and will first instantiate what OpenAI builds and then extend it alongside its own MAI silicon lineage.Satya Nadella — How Microsoft is preparing for AGI
- The depreciating GPU asset is about 75% of the total cost of ownership of a datacenter over five-six years, and Nvidia takes roughly a 75% margin on that, which is why hyperscalers develop their own accelerators to reduce this cost and increase their margins.Satya Nadella — How Microsoft is preparing for AGI
- Because no one knows Nvidia's architecture better than Nvidia, its engineers routinely get partner AI labs another 2x-3x out of their stack or a particular kernel, which—across an installed fleet of Hoppers and Blackwells—directly doubles revenue, so Nvidia's expertise remains needed even for sophisticated customers.Dwarkesh Patel Podcast - Jensen Huang – TPU competition, why we should sell chips to China, & Nvidia’s supply chain moat
- Nvidia's computing stack delivers the best performance per total cost of ownership of any platform, evidenced by competitors (TPU, Trainium) declining to publish results on open benchmarks like InferenceMAX or MLPerf despite claiming cost advantages.Dwarkesh Patel Podcast - Jensen Huang – TPU competition, why we should sell chips to China, & Nvidia’s supply chain moat