A method that separates two groups by finding the widest possible gap between them. Before deep learning it was one of the best general-purpose classifiers available, and it still performs well on small datasets where a neural network would simply memorise.
Technical view
Finds the maximum-margin separating hyperplane, depending only on the support vectors nearest the boundary. The kernel trick allows non-linear boundaries by implicitly mapping into a higher-dimensional space without computing the coordinates. Strong with limited data and high dimensions, but scaling is poor beyond tens of thousands of examples, and probability estimates require an extra calibration step.
Related
Part of AI Terminology Index, section: Classical Machine Learning.