The questioner built neural networks from sub-1-million parameter models, watched them scale to billions (GPT-2 ~1.5B), then to 175 billion (GPT-3), and observed mind-blowing differences in capability at each order of magnitude increase.
factualpending
Evidence Quote
“I've been building neural networks myself since the the parameter count was sub 1 million and they're actually very useful for a bunch of commercial applications and then kind of watch them tip into the billion and then the you know with gpt2 I think about one and a half billion or so uh and then gpt3 and now gpt4 so you go up we don't know the current parameter count but I think it was 175 billion um in gpd3 and it was just mind-blowingly different from gpt2 and then gpt4 is even more mind-blowingly different”
Created: 8/10/2026, 10:46:10 PM
Your notes
Loading notes...