causal
Bigger models without new data memorize, not generalize
Adding parameters without adding new data makes a model memorize rather than generalize, because models pass through a grokking phase where understanding emerges only after repeated exposure in varied forms.
causalpending
Speaker
Dylan PatelEvidence Quote
“if you make the model bigger and bigger without the data changing, you just memorize everything. And actually it starts to get worse again because it never had the opportunity to generalize.”
Created: 11/16/2026, 4:01:45 AM
Your notes
Loading notes...