overfitting
When a model is fine-tuned so closely to past data that it captures noise rather than signal and predicts poorly on new data.
As defined in Gerd Gigerenzer on Gut Feelings
Other definitions
A problem in machine learning where a model learns patterns specific to training data including noise, so it performs well on training data but poorly on new data; in benchmarking, overfitting means models may optimize specifically for benchmark tests
AI Trends 2025: Mary Meeker Deck Deep Dive Part 1
Where it comes up · 15
- Machine Intelligence and Risk Management6 claims
- Susan Athey on Machine Learning, Big Data, and Causation4 claims
- Compound Your Face Off3 claims
- How to Build a Great Product3 claims
- AI Trends 2025: Mary Meeker Deck Deep Dive Part 12 claims
- The Search for the Truth with the Anonymous Master2 claims
- Machine Learning in Investing2 claims
- Gerd Gigerenzer on Gut Feelings2 claims
and 7 more
Most important claims · 10 of 33Search all claims for “overfitting” →
- Econometrics has always prioritized interpretable, reliable models that hold up across settings and reflect how the world works, implicitly sacrificing in-sample predictive power; machine learning is far better at fitting any particular dataset, while econometrics is better when you need robustness to changing environments and protection against spurious relationships.Susan Athey on Machine Learning, Big Data, and Causation
- Overfitting arises both from the model and from the person who believes a strong backtest or forward test is meaningful rather than random, so false confidence in the output is itself a form of overfitting.Machine Intelligence and Risk Management
- When ML teams chase accuracy by adding features, they drift into a long tail of weak features such as camera direction or weather, which may show short-term predictive power but overfit the data and produce downstream distortions.How to Build a Great Product
- The proof of a model is in prediction, not fit; the bias-variance dilemma shows that for prediction it is better to be simpler, because heuristics with few free parameters introduce a bias but reduce error from variance (overfitting), and fine-tuning on the past only pays off if the future resembles the past—true under risk but not under uncertainty.Gerd Gigerenzer on Gut Feelings
- To get valid statistical inference after data-driven model selection, use sample splitting: use one half of the data to choose the model and the other half to estimate the effect; though throwing away data seems horrifying in a small-data world, the gain in functional-form fit justifies it and yields a correct standard error.Susan Athey on Machine Learning, Big Data, and Causation
- Google Flu Trends, hailed in a 2008/2009 Nature article using 45 variables fitted on millions of search terms, failed when the 2009 swine flu peaked in summer because the algorithm had learned flu is high in winter; engineers responded by adding complexity (up to 160 variables) which still failed, illustrating that overfit complex models break under uncertainty.Gerd Gigerenzer on Gut Feelings
- Before throwing data into a prediction system, you need a reasonable hypothesis about why the data should yield a useful prediction; otherwise teams end up adding ever more features from a long tail of weak signals that overfit the data.How to Build a Great Product
- As the number of dimensions (features) grows, distance-based methods like k-nearest-neighbor fail because in high dimensions everything is roughly as far from everything else, so the notion that similar data points are closer than dissimilar ones collapses—this is the curse of dimensionality, predating and orthogonal to neural networks.336 | Anil Ananthaswamy on the Mathematics of Neural Nets and AI
- Unlike classical models, where more parameters overfit and generalize worse, transformers do not fall into this trap as much, so adding more money and compute keeps making them better.Gavin Uberti - Real-Time AI & The Future of AI Hardware - [Invest Like the Best, EP.356]
- Because ML libraries like scikit-learn, XGBoost, and TensorFlow are now freely available at near-zero marginal cost, the scarce resource is no longer access but knowing how to use these tools well; people often misuse them by applying them where they should not and breaking overfitting rules.How to Build a Great Product