A strange phenomenon where a model appears to have simply memorised its training data, then, if you keep training it long after that point, suddenly generalises. It suggests that memorisation and understanding are distinct processes and that one can follow the other, which is unsettling for how training is usually stopped.
Technical view
Observed on small algorithmic tasks, where test accuracy jumps from chance to near-perfect long after training accuracy saturates. Explanations centre on weight decay driving a transition from a memorising to a generalising circuit, with interpretability work identifying the emergence of structured representations at the transition. It is mainly of theoretical interest, and it complicates the assumption that training loss plateauing means learning has finished.
Related
Part of AI Terminology Index, section: Training and Post-training.