When constrained by data but unlimited in compute, aggressive weight decay regularization (30x larger than compute-optimal settings) produces clean power-law scaling with exponent 1 over model parameters and a measurable asymptote representing optimal performance under infinite compute.
factualpending
Speaker
Ku WooEvidence Quote
“you can show that your loss follows a really clean power law... exponent on the model parameters n is one... the scaling law has an asmtote which is 3.43 in this case”
Created: 8/12/2026, 6:25:34 PM
Your notes
Loading notes...