The questioner built neural networks from sub-1-million parameter models, watched them scale to billions (GPT-2 ~1.5B), then to 175 billion (GPT-3), and observed mind-blowing differences in capability at each order of magnitude increase.

factualpending

Speaker

Unidentified Speaker — Breakthrough potential of AI | Sam Altman | MIT 2023 [T5cPoNwO7II]

Evidence Quote

“I've been building neural networks myself since the the parameter count was sub 1 million and they're actually very useful for a bunch of commercial applications and then kind of watch them tip into the billion and then the you know with gpt2 I think about one and a half billion or so uh and then gpt3 and now gpt4 so you go up we don't know the current parameter count but I think it was 175 billion um in gpd3 and it was just mind-blowingly different from gpt2 and then gpt4 is even more mind-blowingly different”

Source

Breakthrough potential of AI | Sam Altman | MIT 2023— Imagination in Action
Created: 8/10/2026, 10:46:10 PM

Your notes

Loading notes...