Deep Seek V3 uses a mixture of experts architecture with 671 billion total parameters but only activates 37 billion parameters for each token prediction, compared to Llama 3's largest model which uses 405 billion parameters without mixture of experts, meaning V3 activates 11x fewer parameters per forward pass.

factualpending

Speaker

Unidentified Speaker — The Engineering Unlocks Behind DeepSeek | YC Decoded [4Tmn-XP93m4]

Evidence Quote

“deep seek V3 has 671 billion modern parameters but only 37 billion are activated for a given token prediction...V3 activates 11x fewer parameters for each forward pass”

Source

The Engineering Unlocks Behind DeepSeek | YC Decoded— Y Combinator
Created: 8/11/2026, 6:17:44 AM

Your notes

Loading notes...