The 7 billion parameter version of DeepSeek R1 distilled Qwen, small enough to run on a decent laptop, still clears 55% on AIME 2024—achieving 1/20th the size of the teacher with most of the capability intact.

factualpending

Speaker

Unidentified Speaker — Model Distillation: How to Steal a Billion-Dollar AI [gacMZf5rqYA]

Evidence Quote

“The 7 billion parameter version... still clears 55% on that same brutal math exam... 1/20 the size of the teacher, most of the capability intact.”

Source

Model Distillation: How to Steal a Billion-Dollar AI— Kai
Created: 8/12/2026, 2:31:09 AM

Your notes

Loading notes...