The result of distilling R1 into small open Qwen models should not be possible on paper—a 1/20th-sized student model maintaining most of the teacher's capability violates conventional scaling expectations.

factualpending

Speaker

Unidentified Speaker — Model Distillation: How to Steal a Billion-Dollar AI [gacMZf5rqYA]

Evidence Quote

“The result should not be possible on paper.”

Source

Model Distillation: How to Steal a Billion-Dollar AI— Kai
Created: 8/12/2026, 2:31:09 AM

Your notes

Loading notes...