Training data set size has been scaling rapidly since 2010 and reached 1e13 (10 trillion) words, but there is a substantial unresolved question about whether data will run out and whether we can continue scaling training data sets at the current log-scale pace without finding alternative scaling laws.

factualpending

Speaker

Nate (Host/Analyst)

Evidence Quote

“I think the question is can we keep scaling from here? Is there a limit? We are one e to the 13 on uh number of words in the data set. Does it make sense to keep going up?”

Source

AI Trends 2025: Mary Meeker Deck Deep Dive Part 1— AI News & Strategy Daily | Nate B Jones
Created: 8/11/2026, 7:29:17 AM

Your notes

Loading notes...