causal

Pre-Training Is a Fixed Cost, Inference Scaling Recurs

The choice between scaling pre-training and inference scaling is an economic one: pre-training is a fixed cost giving the model a permanent capability, while inference scaling costs money per query forever, so whether to spend more on training depends on how many users you have and how long the model stays on the market.

causalpending

Speaker

Sebastian Raschka

Evidence Quote

“It becomes a question of how many users you have and doing the math.”

Source

Ai Sota #2026— Lex Fridman Podcast
Created: 6/13/2026, 3:24:41 AM

Your notes

Loading notes...