An alternative to attention that processes a sequence by maintaining a running summary, rather than looking back at everything each time. That makes cost grow in step with length rather than with its square, which matters enormously for very long inputs. Mamba is the best known example.
Technical view
Derived from continuous-time linear systems, discretised and made input-dependent through selective mechanisms, giving linear-time training via parallel scan and constant-memory inference per token. Competitive with transformers at moderate scale and stronger on very long sequences, but weaker on tasks requiring precise recall of arbitrary earlier content, which is why hybrid architectures interleaving a few attention layers are the common compromise.
Related
Part of AI Terminology Index, section: How the Machinery Works.