When constrained by data but unlimited in compute, aggressive weight decay regularization (30x larger than compute-optimal settings) produces clean power-law scaling with exponent 1 over model parameters and a measurable asymptote representing optimal performance under infinite compute.

factualpending

Speaker

Ku Woo

Evidence Quote

“you can show that your loss follows a really clean power law... exponent on the model parameters n is one... the scaling law has an asmtote which is 3.43 in this case”

Source

Inference, Diffusion, World Models, and More | YC Paper Club— Y Combinator
Created: 8/12/2026, 6:25:34 PM

Your notes

Loading notes...