definition
Pretrained Models Are Crude Oil
Pretrained models can be understood as unrefined crude oil; their real capabilities only become discoverable once they are instruction-tuned and RLHF'd and deployed in usable formats—as shown by GPT-3, whose quality wasn't appreciated until instruction tuning produced ChatGPT.
definitionpending
Speaker
Nat Friedman (Github CEO)Evidence Quote
“people didn't know how good GPT-3 was until it got instruction tuned for chatGPT and was put out in that format.”
Source
Nat Friedman (Github CEO) — Reading ancient scrolls, open source, & AI— Dwarkesh Patel PodcastCreated: 6/13/2026, 3:32:28 AM
Your notes
Loading notes...