In model distillation step three, DeepSeek can record additional details from GPT's intermediate layers including attention scores and hidden states—essentially accessing the teacher's reasoning notes alongside the final answer.
factualpending
Speaker
Dong SeanEvidence Quote
“deep SE can gather other details from the model's intermediate layers like attention scores or hidden States it's like getting access to the teacher's notes”
Created: 8/11/2026, 7:36:57 AM
Your notes
Loading notes...