Model Distillation

Definition

Technique for compressing a large, expensive neural network into a smaller, cheaper version by identifying the most important parameters and knowledge, then training a smaller model to mimic the larger one's behavior. Results in most of the capability at a fraction of the compute cost.

Cast within

No topic-region cast yet — this appears once Model Distillation's compiled claims are aligned into a topic region's argument tree.

Claims mentioning Model Distillation (9)

Your notes

Loading notes...