Sheela
Sheela
Home
Add
Questions
Thinkers
Claims
People
Concepts
Jargon
Sources
Health
Predictions
Collections
Domain Landscape
Contribute
Review
Dedup
Topic requests
Settings
Help

← Jargon

Jargonmachine learning· 2 claims· 2 episodes

double descent

A non-monotonic phenomenon in machine learning where test error exhibits a U-shaped curve as model capacity increases: performance improves initially, worsens sharply around the interpolation threshold (where parameters equal training data), then improves again with further scaling.

As defined in Ilya Sutskever: Deep Learning | Lex Fridman Podcast #94


Where it comes up · 2
  • Ilya Sutskever: Deep Learning | Lex Fridman Podcast #941 claim
  • Yann LeCun: Special Lecture on AI and World Models1 claim

Most important claims · 2 of 2Search all claims for “double descent” →
  • Double descent is a phenomenon where test error exhibits a non-monotonic U-shape as model size increases while data is fixed: performance improves as networks grow, then worsens sharply around the interpolation threshold (where model parameters equal training examples), then improves again as model size continues to grow, and this pattern occurs across virtually all practical deep learning systems without early stopping.Ilya Sutskever: Deep Learning | Lex Fridman Podcast #94
  • Double descent is an empirical phenomenon where test error first increases then decreases as model size increases beyond the interpolation threshold, contradicting classical statistical intuition that overfitting increases with model capacity.Yann LeCun: Special Lecture on AI and World Models