factual
Infinite parameters cut data need only tenfold
Under the Chinchilla scaling constants, where parameter and data loss terms add independently, even infinitely many parameters would reduce the data needed for a given loss by only about a factor of 10, and the result would not change with different constants, so scaling cannot close a gap of thousands to millions of times.
factualpending
Speaker
Dwarkesh PatelEvidence Quote
“even if you increased the number of parameters by infinity, that would only decrease by a factor of ~10 the amount of data you need”
Created: 11/17/2026, 6:11:50 AM
Your notes
Loading notes...