Jargonautica
Retrieval and Knowledge

BM25 and Sparse Retrieval

Also called: BM25, Sparse Retrieval

The classic keyword ranking method, refined over decades and still very hard to beat. It scores documents by how often the query words appear, adjusted for how common those words are generally and how long the document is. It is fast, needs no training, and explains its own results.

Technical view

A probabilistic ranking function using term frequency saturation and inverse document frequency with length normalisation, with tunable parameters controlling saturation and length effects. Implemented on inverted indexes, giving low latency at very large scale. Learned sparse methods such as SPLADE expand queries and documents into weighted term sets, capturing some semantic matching while keeping the efficiency of an inverted index.


Part of AI Terminology Index, section: Retrieval and Knowledge.

How it connects

BM25 and Sparse Retrieval links to 6 other terms. The faint lines are links between those terms. See it in the full graph.

Semantic SearchSemantic SearchHybrid SearchHybrid SearchInformation RetrievalInformation Retriev…Inverted IndexInverted IndexRetrieval MetricsRetrieval MetricsVector DatabaseVector DatabaseBM25 and Sparse Retrieval