In black box distillation, only text outputs are visible; prompts are sent, answers are saved, and small models are trained on those outputs alone, which is a weaker signal than soft labels but repeated across 100,000 examples burns the teacher's habits into the student.
factualpending
Evidence Quote
“In black box distillation, you see nothing inside... even a weak signal burns the teacher's habits into the student.”
Created: 8/12/2026, 2:31:09 AM
Your notes
Loading notes...