Training data set size has been scaling rapidly since 2010 and reached 1e13 (10 trillion) words, but there is a substantial unresolved question about whether data will run out and whether we can continue scaling training data sets at the current log-scale pace without finding alternative scaling laws.
factualpending
Speaker
Nate (Host/Analyst)Evidence Quote
“I think the question is can we keep scaling from here? Is there a limit? We are one e to the 13 on uh number of words in the data set. Does it make sense to keep going up?”
Created: 8/11/2026, 7:29:17 AM
Your notes
Loading notes...