The idea that real-world data, such as photographs or speech, occupies only a tiny, smoothly shaped region of all the combinations it could in principle take. Almost every random arrangement of pixels is noise, not a picture. If meaningful data lies on a much simpler surface hidden inside that vast space, a model only has to learn the shape of that surface, which helps explain why learning from examples works at all.
Technical view
Posits that high-dimensional natural data concentrates near low-dimensional manifolds embedded in the ambient space, with local coordinates corresponding to factors of variation such as pose or lighting. It offers an escape from the curse of dimensionality, since, under that assumption, theory suggests sample complexity can depend on intrinsic rather than ambient dimension, and it motivates autoencoders, nonlinear dimensionality reduction and the view of deep networks as progressively flattening data manifolds. Latent space interpolation and adversarial examples, which step off the manifold, are often explained in its terms. It is an empirical hypothesis: real data may be a union of manifolds of varying dimension.
Related
Part of AI Terminology Index, section: The Foundations.