The mechanism that lets a model decide which earlier words matter most when working out what comes next. In the sentence ‘the dog chased the ball because it was fast’, attention is what connects ‘it’ to ‘the dog’ rather than to ‘the ball’. It gives the model a way to weigh relevance rather than treating all preceding words equally.
Technical view
Each token produces three vectors: a query, a key and a value. Relevance between tokens is computed as the dot product of one token’s query with another’s key, scaled and passed through a softmax to produce weights, which are then used to take a weighted sum of value vectors. Models use multiple attention heads in parallel, each free to specialise in different relationships, such as syntactic dependencies or coreference. Interpretability research has found individual heads performing surprisingly specific and identifiable functions.
Related
Part of AI Terminology Index, section: Language Models Specifically.