Training dataset size may be hitting scaling limits around 10^13 words, with Ilya Sutskever (who founded Safe Super Intelligence) claiming data is becoming the limiting factor, suggesting future AI scaling will need to rely on algorithmic improvements and better chip utilization rather than raw data growth.
factualpending
Evidence Quote
“We are one e to the 13 on uh number of words in the data set... Uliagiver gave a talk at Nurips last year basically saying data is the new oil. we don't have an infinite sort of amount of data left.”
Created: 8/11/2026, 7:38:24 AM
Your notes
Loading notes...