Sheela
Sheela
Home
Add
Questions
Thinkers
Claims
People
Concepts
Jargon
Sources
Health
Predictions
Collections
Domain Landscape
Contribute
Review
Dedup
Topic requests
Settings
Help

← Jargon

Jargonmachine learning· 1 claim· 1 episode

activated parameters

In mixture of experts models, the number of parameters in the expert(s) that are actually used to process a given input, as opposed to total parameters in all experts

As defined in Microsoft Goes FULL NUCLEAR to Develop AGI...


Where it comes up · 1
  • Microsoft Goes FULL NUCLEAR to Develop AGI...1 claim

Most important claims · 1 of 1Search all claims for “activated parameters” →
  • Grok Moe is a mixture of experts model with 16 experts and 3.8 billion total parameters, with 6.6 billion activated parameters, that outperforms a 7 billion parameter dense model and matches the performance of a 14 billion parameter dense model on the same training data.Microsoft Goes FULL NUCLEAR to Develop AGI...