Jargonautica
Language Models Specifically

Logits and Softmax

Also called: Logits, Softmax

What the model actually produces before you see any text. At each step it emits a raw score for every possible next token, tens of thousands of them. Softmax turns those scores into probabilities that add up to one, and only then does something choose a token. Temperature and the various sampling controls all operate at this step, which is why they change style and reliability without changing the model at all.

Technical view

Logits are the unnormalised outputs of the final linear layer. Softmax exponentiates and normalises them, and dividing logits by a temperature beforehand flattens or sharpens the distribution. Log probabilities of chosen tokens are exposed by many interfaces and serve as a rough confidence signal, though they measure fluency rather than truth and are poorly calibrated after preference training. Constrained decoding works by masking the logits of any token that would break a required schema.


Part of AI Terminology Index, section: Language Models Specifically.

How it connects

Logits and Softmax links to 10 other terms. The faint lines are links between those terms. See it in the full graph.

Temperature and SamplingTemperature and Sam…TokensTokensStructured OutputStructured OutputPerplexityPerplexityProbability DistributionProbability Distrib…Logit BiasLogit BiasModel Confidence and CalibrationModel Confidence an…Activation FunctionActivation FunctionAttention SinkAttention SinkMulti-label and Ordinal ClassificationMulti-label and Ord…Logits and Softmax