Pre-training Scaling Laws
Definition
The earlier framework that model capability rises predictably with more pre-training compute, data, and parameters, implying that 10x more compute yields a step-function gain. It is now seen as plateauing because human text data and synthetic-data substitutes are insufficient.
Cast within
No topic-region cast yet — this appears once Pre-training Scaling Laws's compiled claims are aligned into a topic region's argument tree.
Claims mentioning Pre-training Scaling Laws (2)
Your notes
Loading notes...