Jargonautica
Maths and Statistics

Curse of Dimensionality

Things get strange when data has many columns. Points spread out, everything becomes roughly equally far from everything else, and the amount of data needed to cover the space grows impossibly fast. It is the reason simple distance-based methods stop working, and part of why learned representations that compress dimensions are so valuable.

Technical view

As dimension grows, volume concentrates near the boundary of a region, pairwise distances converge in relative terms, and the sample size needed for a given density grows exponentially. This degrades nearest-neighbour methods and kernel density estimation. Deep learning works despite it because real data lies near a much lower-dimensional manifold, and learned representations approximate that manifold rather than the ambient space.


Part of AI Terminology Index, section: Maths and Statistics.

How it connects

Curse of Dimensionality links to 8 other terms. The faint lines are links between those terms. See it in the full graph.

EmbeddingsEmbeddingsDimensionality ReductionDimensionality Redu…VectorVectorManifold HypothesisManifold HypothesisK-Nearest NeighboursK-Nearest NeighboursSupport Vector MachineSupport Vector Mach…Monte Carlo MethodsMonte Carlo MethodsInterpolation and ExtrapolationInterpolation and E…Curse of Dimensionality