causal

Pre-Training Is Compute-Bound by Serving Costs

Pre-training has gotten extremely expensive and scaling it implies serving a very large model, but the cost of serving millions of users (billions of dollars of compute) dwarfs the relatively low training cost (DeepSeek's ~$5M, OLMo 3's ~$2M cluster rental), so labs shrink models to reduce serving costs even though bigger models would be smarter.

causalpending

Speaker

Nathan Lambert

Evidence Quote

“the recurring costs of serving millions of users is really billions of dollars of compute.”

Source

Ai Sota #2026— Lex Fridman Podcast
Created: 6/13/2026, 3:24:41 AM

Your notes

Loading notes...