The finding that as models get bigger, performance gets worse for a while and then gets better again, contradicting the textbook rule that more capacity means more overfitting. It is a large part of why the field stopped worrying about model size and kept scaling.
Technical view
Test error follows a classical bias-variance U-curve up to the interpolation threshold, where the model exactly fits the training data, spikes there, then decreases again in the overparameterised regime. Observed with respect to model size, data size and training time. The prevailing explanation is implicit regularisation: among the many parameter settings that fit the data, gradient descent finds ones with favourable properties.
Related
Part of AI Terminology Index, section: Training and Post-training.