factual
Pre-training scaling laws are empirical, not understood
Scaling laws for pre-training are an empirical observation measured very precisely and stable over time, but no one knows how or why they work, so they should be treated as a law-like regularity without a theory.
factualpending
Speaker
Gavin BakerEvidence Quote
“no one on planet Earth knows how or why scaling laws for pre-training work. It's actually not a law. It's an empirical observation, and it's an empirical observations that we've measured extremely precisely and is held for a long time.”
Created: 11/20/2026, 2:50:27 AM
Your notes
Loading notes...