In black box distillation, only text outputs are visible; prompts are sent, answers are saved, and small models are trained on those outputs alone, which is a weaker signal than soft labels but repeated across 100,000 examples burns the teacher's habits into the student.

factualpending

Speaker

Unidentified Speaker — Model Distillation: How to Steal a Billion-Dollar AI [gacMZf5rqYA]

Evidence Quote

“In black box distillation, you see nothing inside... even a weak signal burns the teacher's habits into the student.”

Source

Model Distillation: How to Steal a Billion-Dollar AI— Kai
Created: 8/12/2026, 2:31:09 AM

Your notes

Loading notes...