factual

Pre-training scaling plateau after synthetic data limits

All major labs have hit a plateau in pre-training scaling over the last two years, because the world's human-generated text data has been largely tokenized and models trained on synthetic data are not enabling scaling to continue.

factualpending

Speaker

Chetan

Evidence Quote

“The synthetic data as generated by the LLMs themselves are not enabling the scaling in pre-training to continue.”

Source

Capital, Compute & AI Scaling— Invest Like the Best
Created: 11/18/2026, 6:42:24 AM

Your notes

Loading notes...