Pre-training Scaling Laws

Definition

The earlier framework that model capability rises predictably with more pre-training compute, data, and parameters, implying that 10x more compute yields a step-function gain. It is now seen as plateauing because human text data and synthetic-data substitutes are insufficient.

Cast within

No topic-region cast yet — this appears once Pre-training Scaling Laws's compiled claims are aligned into a topic region's argument tree.

Claims mentioning Pre-training Scaling Laws (2)

Your notes

Loading notes...