Scaling laws in neural networks are empirically demonstrated as deep and fundamental truths, yet lack complete theoretical explanation—neural networks designed in the 1940s still show consistent capability improvements with increasing compute without hitting a performance wall.
factualpending
Speaker
Greg BrockmanEvidence Quote
“the scaling laws are a deep and very beautiful mystery... they're empirical. Like we don't necessarily have all the theory to explain exactly why it works... neural networks were really designed like in the 1940s... as you pour more compute into the models, they get correspondingly more capable and it just keeps going. There's no wall.”
Created: 8/12/2026, 6:18:53 PM
Your notes
Loading notes...