Model Distillation
Definition
Technique for compressing a large, expensive neural network into a smaller, cheaper version by identifying the most important parameters and knowledge, then training a smaller model to mimic the larger one's behavior. Results in most of the capability at a fraction of the compute cost.
Cast within
No topic-region cast yet — this appears once Model Distillation's compiled claims are aligned into a topic region's argument tree.
Claims mentioning Model Distillation (9)
Your notes
Loading notes...