factual
ChatGPT was a cheap unhobbling of pretrained models
ChatGPT exemplifies cheap 'unhobbling': roughly 1% of additional compute spent on post-training a pretrained internet corpus was enough to make a model highly competent at chatbot conversation, and reasoning similarly emerged from a small fraction of compute spent on RL.
factualpending
Speaker
Dwarkesh PatelEvidence Quote
“1% of additional compute spent on getting it to talk in a chatbot-like fashion with post training is enough to make it competent”
Created: 6/13/2026, 3:24:17 AM
Your notes
Loading notes...