Removing parts of a model that contribute little, in the same way you might cut dead branches off a tree. Large models turn out to contain a lot of near-redundant machinery, so a surprising proportion can often be cut with little loss of quality.
Technical view
Unstructured pruning zeroes individual weights below some importance threshold, producing sparse matrices that need specialised hardware support to yield real speed gains. Structured pruning removes whole units, attention heads or layers, which gives immediate practical speed-ups on standard hardware. Pruning is usually followed by a short retraining phase to recover lost accuracy. It is often combined with quantisation, since the two address different costs.
Related
Part of AI Terminology Index, section: Making Models Smaller, Faster and Cheaper.