factual

Scaling law extrapolation is imperfect information

Researchers operate under imperfect information because scaling-law trends never reliably hold: you fit a curve through small-scale dots to estimate the final model's performance, but things that help at small scale can hurt at large scale, so you must guess based on trend lines and intuition about what will actually matter at scale.

factualpending

Speaker

Sholto Douglas

Evidence Quote

“things which can help at smaller scales can actually hurt at larger scales. You have to make guesses based on what the trend lines look like and based on your intuitive feeling of what's actually something that's going to matter”

Source

Sholto Douglas & Trenton Bricken — How LLMs actually think— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:31:11 AM

Your notes

Loading notes...