causal
Pre-Training Is Compute-Bound by Serving Costs
Pre-training has gotten extremely expensive and scaling it implies serving a very large model, but the cost of serving millions of users (billions of dollars of compute) dwarfs the relatively low training cost (DeepSeek's ~$5M, OLMo 3's ~$2M cluster rental), so labs shrink models to reduce serving costs even though bigger models would be smarter.
causalpending
Speaker
Nathan LambertEvidence Quote
“the recurring costs of serving millions of users is really billions of dollars of compute.”
Created: 6/13/2026, 3:24:41 AM
Your notes
Loading notes...