Mixture of Experts

Definition

An architecture where a neural network comprises multiple specialized subnetworks (experts), each trained on different types of inputs or problems, with a gating mechanism intelligently routing each input to the most appropriate expert(s). This enables more efficient scaling than monolithic models because only a fraction of parameters are activated per input.

Cast within

No topic-region cast yet — this appears once Mixture of Experts's compiled claims are aligned into a topic region's argument tree.

Claims mentioning Mixture of Experts (7)

Your notes

Loading notes...