Jargonautica
Classical Machine Learning

K-Nearest Neighbours

Also called: KNN, Nearest Neighbours

Predict by looking at the most similar examples you have already seen and taking their answer. There is no training as such: the model is the data. It is the simplest thing that works, which makes it the right first baseline before anyone builds something clever.

Technical view

A non-parametric, instance-based method where the choice of k trades bias against variance and the distance metric determines everything. Prediction cost grows with dataset size, mitigated by approximate nearest neighbour indexes, which is the same machinery vector databases use. It degrades badly in high dimensions and requires feature scaling, but retrieval-augmented generation is recognisably its descendant.


Part of AI Terminology Index, section: Classical Machine Learning.

How it connects

K-Nearest Neighbours links to 7 other terms. The faint lines are links between those terms. See it in the full graph.

Vector DatabaseVector DatabaseRetrieval Augmented GenerationRetrieval Augmented…Approximate Nearest Neighbour SearchApproximate Nearest…Curse of DimensionalityCurse of Dimensiona…Support Vector MachineSupport Vector Mach…BaselineBaselineNormalisation and StandardisationNormalisation and S…K-Nearest Neighbours