In model distillation step two, GPT generates 'soft labels' after analyzing a sentence—a probability distribution showing, for example, an 85% chance the subject is a cat and 15% chance it's a mat, which not only tells DeepSeek the right answer but provides insight into how GPT reasoned to that conclusion.

factualpending

Speaker

Dong Sean

Evidence Quote

“step two gbt generates a soft labels after analyzing the sentence...it might say there is an 85% chance the subject is a cat and 15% that it's a mat”

Source

The AI Shortcut: How DeepSeek Cracked ChatGPT’s Secrets for Cheap— Digging into China
Created: 8/11/2026, 7:36:57 AM

Your notes

Loading notes...