causal

Going sparser is a pure win given enough users

From a pure bandwidth-time perspective, increasing sparsity is a win as long as you have enough users to fill larger batches, because the doubled weight-memory fetches are amortized away by running a bigger batch; the binding cost is instead the additional memory capacity the larger total parameter count consumes.

causalpending

Speaker

Reiner Pope

Evidence Quote

“if I have a lot of users, I can go to a much sparser model. From that point of view, it’s a reasonable trade-off. The other trade-off that shows up here is that it also consumes memory capacity.”

Source

Dwarkesh Patel Podcast - Reiner Pope – The math behind how LLMs are trained and served— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:34:09 AM

Your notes

Loading notes...