The mystery of overparameterization is resolved by Pack-Bayes theory: as model parameters increase, empirical risk (training loss) decreases because models fit data better, and simultaneously, more compressible solutions are found, so both terms of the Pack-Bayes bound decrease.
causalpending
Speaker
AsheEvidence Quote
“When you increase the number of parameters you can fit your data better... the empirical risk the left uh the first term goes down... when we increase the number of parameters, um we also find more compressible solutions.”
Created: 8/12/2026, 6:25:34 PM
Your notes
Loading notes...