curve fitting
The process of fitting a smooth parametric function (curve) to data points so it interpolates between them; Chollet's characterization of how deep learning models learn.
As defined in 280 | François Chollet on Deep Learning and the Meaning of Intelligence
Where it comes up · 2
- 280 | François Chollet on Deep Learning and the Meaning of Intelligence3 claims
- Crowdsourcing Predictive Algorithms1 claim
Most important claims · 4 of 4Search all claims for “curve fitting” →
- Machine learning is essentially regression and curve fitting, and a fitted model can predict well even when the analyst does not know what the input axes represent, as with Numerai's roughly 50 anonymous dimensions.Crowdsourcing Predictive Algorithms
- ARC (Abstraction and Reasoning Corpus for AGI), released in 2019, is a benchmark of unique, original puzzles never seen before and partly kept private, designed so that memorization cannot help; humans solve them easily while deep learning and LLMs score very poorly—GPT-3 scored zero in the 2020 Kaggle competition—because each puzzle is genuinely novel and LLMs have essentially no ability to adapt to such novelty.280 | François Chollet on Deep Learning and the Meaning of Intelligence
- Because LLMs are big differentiable parametric curves fit to a data distribution, you cannot input symbolic rules valid for whole categories of problems; you can only supply individual data points the model interpolates between, so fixing errors requires labor-intensive pointwise patches that address one query at a time rather than entire categories of issues.280 | François Chollet on Deep Learning and the Meaning of Intelligence
- Curve fitting via gradient descent is a form of learning with non-zero but very low intelligence because it converts data into solutions at an extremely poor, data-inefficient conversion ratio; a truly intelligent system would let you fix an error by explaining why an answer was wrong and have it automatically patch the entire underlying category of issues, rather than requiring pointwise fixes.280 | François Chollet on Deep Learning and the Meaning of Intelligence