factual

Scaling Foreseen By Watching Neural Nets Broaden

Gwern foresaw scaling not via a eureka moment but by repeatedly noticing, over years, that dataset sizes, model sizes, and GPU counts kept growing while neural networks kept generalizing to ever-broader use cases, gradually shifting his prior away from the view that algorithms and deep insight are paramount toward the view that intelligence is mostly compute applied to data and parameters.

factualpending

Speaker

Gwern

Evidence Quote

“I just keep reading deep learning literature and noticing again and again that the dataset size keeps getting bigger. The models keep getting bigger.”

Source

Gwern — Anonymous writer who predicted AI trajectory on $12K/year salary— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:28:51 AM

Your notes

Loading notes...