Mixture of experts architecture is not new but has been challenging to train models with this architecture efficiently; DeepSeek introduced novel techniques that stabilize performance and increase GPU utilization in mixture of experts training

factualpending

Speaker

Unidentified Speaker — The Engineering Unlocks Behind DeepSeek | YC Decoded [4Tmn-XP93m4]

Evidence Quote

“mixture of experts isn't A New Concept but it's been challenging to train models with this architecture efficiently deep seek introduced novel techniques that stabilize performance and increase GPU utilization”

Source

The Engineering Unlocks Behind DeepSeek | YC Decoded— Y Combinator
Created: 8/11/2026, 6:44:59 AM

Your notes

Loading notes...