Jargonautica
Controlling and Extending Behaviour at Run Time

Self-consistency

Asking the same question several times and taking the most common answer. Because the model samples differently each time, mistakes tend to be scattered while correct answers converge. It costs several times as much and reliably improves accuracy on reasoning problems.

Technical view

Samples multiple chains of thought at non-zero temperature and marginalises over reasoning paths by majority vote on the final answer. Gains are largest where an answer can be extracted and compared exactly, such as arithmetic or multiple choice, and awkward for open-ended text where votes must be clustered semantically. It also provides a usable confidence signal: agreement rate across samples correlates with correctness better than the model’s stated confidence.


Part of AI Terminology Index, section: Controlling and Extending Behaviour at Run Time.

How it connects

Self-consistency links to 5 other terms. The faint lines are links between those terms. See it in the full graph.

Chain of ThoughtChain of ThoughtTemperature and SamplingTemperature and Sam…Cost Per TaskCost Per TaskModel Confidence and CalibrationModel Confidence an…Monte Carlo MethodsMonte Carlo MethodsSelf-consistency