factual

Frontier MoE models are highly sparse

Few modern MoE models are more than 10 percent dense, and frontier models activate closer to 1 percent of experts, so compute is already used fairly judiciously and is not the main source of waste.

factualpending

Speaker

Neil Movva

Evidence Quote

“I don't think that we're wasting too much on the MoE side.”

Source

Neil Movva - Making AI 10x Cheaper - [Invest Like the Best, EP.488]— Invest Like the Best
Created: 11/17/2026, 6:43:49 AM

Your notes

Loading notes...