The mystery of overparameterization is resolved by Pack-Bayes theory: as model parameters increase, empirical risk (training loss) decreases because models fit data better, and simultaneously, more compressible solutions are found, so both terms of the Pack-Bayes bound decrease.

causalpending

Speaker

Ashe

Evidence Quote

“When you increase the number of parameters you can fit your data better... the empirical risk the left uh the first term goes down... when we increase the number of parameters, um we also find more compressible solutions.”

Source

Inference, Diffusion, World Models, and More | YC Paper Club— Y Combinator
Created: 8/12/2026, 6:25:34 PM

Your notes

Loading notes...