In model distillation step three, DeepSeek can record additional details from GPT's intermediate layers including attention scores and hidden states—essentially accessing the teacher's reasoning notes alongside the final answer.

factualpending

Speaker

Dong Sean

Evidence Quote

“deep SE can gather other details from the model's intermediate layers like attention scores or hidden States it's like getting access to the teacher's notes”

Source

The AI Shortcut: How DeepSeek Cracked ChatGPT’s Secrets for Cheap— Digging into China
Created: 8/11/2026, 7:36:57 AM

Your notes

Loading notes...