causal
Going sparser is a pure win given enough users
From a pure bandwidth-time perspective, increasing sparsity is a win as long as you have enough users to fill larger batches, because the doubled weight-memory fetches are amortized away by running a bigger batch; the binding cost is instead the additional memory capacity the larger total parameter count consumes.
causalpending
Speaker
Reiner PopeEvidence Quote
“if I have a lot of users, I can go to a much sparser model. From that point of view, it’s a reasonable trade-off. The other trade-off that shows up here is that it also consumes memory capacity.”
Source
Dwarkesh Patel Podcast - Reiner Pope – The math behind how LLMs are trained and served— Dwarkesh Patel PodcastCreated: 6/13/2026, 3:34:09 AM
Your notes
Loading notes...