Jargonautica
The Foundations

No Free Lunch Theorem

The result that no learning method is best at everything. Averaged across all possible problems, every approach performs the same, so any method that does well on the problems we care about does so by making assumptions that happen to fit them. It is the formal reason to be sceptical of claims that one technique is universally superior.

Technical view

Wolpert and Macready’s result concerns performance averaged over all possible objective functions, a set dominated by structureless ones that never occur in practice. The useful reading is not that method choice is arbitrary but that it is an empirical question about which inductive biases match your domain. It is frequently invoked more loosely than the theorem supports.


Part of AI Terminology Index, section: The Foundations.

How it connects

No Free Lunch Theorem links to 4 other terms. The faint lines are links between those terms. See it in the full graph.

Inductive BiasInductive BiasHeuristicHeuristicMachine LearningMachine LearningBaselineBaselineNo Free Lunch Theorem