Jargonautica
Language Models Specifically

Greedy Decoding

Always taking the single most likely next token. It gives the same answer every time for the same input, which sounds ideal and produces flat, repetitive text on anything open-ended. Fine for extraction and classification, poor for writing.

Technical view

Equivalent to temperature zero. Deterministic in principle, though floating-point non-determinism in batched GPU execution means identical inputs can still produce different outputs in practice. Prone to degenerate repetition on long generations, since locally optimal choices compound. It remains the right default for structured output, classification and any task where reproducibility matters more than variety.


Part of AI Terminology Index, section: Language Models Specifically.

How it connects

Greedy Decoding links to 5 other terms. The faint lines are links between those terms. See it in the full graph.

Temperature and SamplingTemperature and Sam…Structured OutputStructured OutputRepetition PenaltyRepetition PenaltyDeterminism and SeedDeterminism and SeedBeam SearchBeam SearchGreedy Decoding