Letting information skip past a layer and be added back on the other side. It sounds like a small architectural detail and it is the reason networks can be hundreds of layers deep. Each layer only has to learn a correction to what it received, rather than reconstruct the whole signal.
Technical view
Introduced in ResNet, computing the output as the input plus the layer’s transformation. This creates a direct path for gradients to flow backwards without repeated multiplication, addressing vanishing gradients, and reframes each block as learning a residual function. Every transformer block contains two of these, and the resulting residual stream is the central object of much interpretability research.
Related
Part of AI Terminology Index, section: How the Machinery Works.