definition

Pretrained Models Are Crude Oil

Pretrained models can be understood as unrefined crude oil; their real capabilities only become discoverable once they are instruction-tuned and RLHF'd and deployed in usable formats—as shown by GPT-3, whose quality wasn't appreciated until instruction tuning produced ChatGPT.

definitionpending

Speaker

Nat Friedman (Github CEO)

Evidence Quote

“people didn't know how good GPT-3 was until it got instruction tuned for chatGPT and was put out in that format.”

Source

Nat Friedman (Github CEO) — Reading ancient scrolls, open source, & AI— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:32:28 AM

Your notes

Loading notes...