Reformulating attention so its cost grows in proportion to the length of the input rather than with the square of it. The quadratic cost of standard attention is the single biggest constraint on context length, so this is a heavily researched area with a long history of approaches that look good in papers and lose quality in practice.
Technical view
Replaces the softmax with a kernel feature map, allowing the associativity of matrix multiplication to be reordered so that the sequence-length-squared term disappears, and enabling a recurrent formulation at inference. The persistent difficulty is that the softmax provides sharp, selective retrieval which kernel approximations blur, degrading exact recall tasks. Recent gated variants narrow the gap and the family overlaps substantially with state space models.
Related
Part of AI Terminology Index, section: How the Machinery Works.