factual
GPT-3 Few-Shot Chart Confirmed The Scaling World
GPT-3 was the crucial test of the scaling hypothesis: a massive scale-up from GPT-2 on a non-narrow task, where unimpressive results would have falsified scaling and impressive ones would confirm it; the few-shot learning chart on the first pages confirmed for Gwern that we live in the 'scaling world' where Legg, Moravec and Kurzweil were right.
factualpending
Speaker
GwernEvidence Quote
“If scaling was bogus, then the GPT-3 paper should just be unimpressive and wouldn't show anything important. Whereas if scaling was true, you would just automatically be guaranteed to get so much more impressive results out of it than GPT-2.”
Source
Gwern — Anonymous writer who predicted AI trajectory on $12K/year salary— Dwarkesh Patel PodcastCreated: 6/13/2026, 3:28:51 AM
Your notes
Loading notes...