causal
Inference batching favors large organizations
Continual learning may create inference economies of scale because per-company information requires full weight updates rather than low-rank adapters; a sparse model like DeepSeek V3 may need an optimal inference batch size above 2,400 sequences, so individual users serving themselves at batch size 1 could suffer a 100x+ compute efficiency penalty, favoring big organizations.
causalpending
Speaker
Dwarkesh PatelEvidence Quote
“So the economics of serving personalized weights strongly favor big organizations.”
Source
Dwarkesh Patel Podcast - 8 Predictions for the Era of Continual Learning— Dwarkesh Patel PodcastCreated: 11/17/2026, 6:25:01 AM
Your notes
Loading notes...