manifold hypothesis
The principle that all natural data in perception tasks lies on a low-dimensional manifold within the high-dimensional space where it is encoded, enabling interpolation between training samples.
As defined in Francois Chollet | Why abstraction is the key to intelligence, and what we’re still missing
Where it comes up · 1
Most important claims · 3 of 3Search all claims for “manifold hypothesis” →
- Within a manifold, it is always possible to interpolate between two inputs by morphing one into another via a continuous path along which all points fall on the manifold; the ability to interpolate between samples is the key to understanding generalization in deep learning because it allows making sense of points never seen by relating them to nearby points on the manifold.Francois Chollet | Why abstraction is the key to intelligence, and what we’re still missing
- The power to generalize in deep learning is much more a consequence of the natural structure of your data than a property of your model; you will only be able to generalize if your data follows the manifold hypothesis, and the more informative and less noisy your data, the better you can generalize.Francois Chollet | Why abstraction is the key to intelligence, and what we’re still missing
- The manifold hypothesis states that all natural data lies on a low-dimensional manifold within the high-dimensional space where it is encoded; this is accurate for any perception problem including MNIST digits, human faces, voice sounds, and natural language.Francois Chollet | Why abstraction is the key to intelligence, and what we’re still missing