Deep Seek V3 uses a mixture of experts architecture with 671 billion total parameters but only activates 37 billion parameters for each token prediction, compared to Llama 3's largest model which uses 405 billion parameters without mixture of experts, meaning V3 activates 11x fewer parameters per forward pass.
factualpending
Evidence Quote
“deep seek V3 has 671 billion modern parameters but only 37 billion are activated for a given token prediction...V3 activates 11x fewer parameters for each forward pass”
Created: 8/11/2026, 6:17:44 AM
Your notes
Loading notes...