factual

GPT-3 Few-Shot Chart Confirmed The Scaling World

GPT-3 was the crucial test of the scaling hypothesis: a massive scale-up from GPT-2 on a non-narrow task, where unimpressive results would have falsified scaling and impressive ones would confirm it; the few-shot learning chart on the first pages confirmed for Gwern that we live in the 'scaling world' where Legg, Moravec and Kurzweil were right.

factualpending

Speaker

Gwern

Evidence Quote

“If scaling was bogus, then the GPT-3 paper should just be unimpressive and wouldn't show anything important. Whereas if scaling was true, you would just automatically be guaranteed to get so much more impressive results out of it than GPT-2.”

Source

Gwern — Anonymous writer who predicted AI trajectory on $12K/year salary— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:28:51 AM

Your notes

Loading notes...