The same idea as batch normalisation, but rescaling across the features of each individual example rather than across a batch. Because it does not depend on what else is in the batch, it behaves identically during training and use, which suits language models.
Technical view
Normalises across the feature dimension per token, independent of batch composition. Modern transformers place it before each sub-layer rather than after, which stabilises training of very deep stacks. RMSNorm, which drops the mean-centring and keeps only the scaling, is now common because it is cheaper and works about as well.
Related
Part of AI Terminology Index, section: How the Machinery Works.