factual
Scaling law extrapolation is imperfect information
Researchers operate under imperfect information because scaling-law trends never reliably hold: you fit a curve through small-scale dots to estimate the final model's performance, but things that help at small scale can hurt at large scale, so you must guess based on trend lines and intuition about what will actually matter at scale.
factualpending
Speaker
Sholto DouglasEvidence Quote
“things which can help at smaller scales can actually hurt at larger scales. You have to make guesses based on what the trend lines look like and based on your intuitive feeling of what's actually something that's going to matter”
Created: 6/13/2026, 3:31:11 AM
Your notes
Loading notes...