Label Noise

Errors in training or evaluation labels, either random (annotator slips) or systematic (consistent confusion between two classes, biased guidelines). Deep networks can memorise noisy labels outright, which harms generalisation. Mitigations include multiple annotators with adjudication, confident learning methods that flag likely mislabels where a model’s out-of-sample predicted probabilities confidently contradict the given label, robust loss functions, and early stopping before memorisation sets in. Noise in test sets is particularly damaging because it distorts the measurement itself: audits of widely used benchmarks have found mislabelled items, so small differences between models near the top of a leaderboard may sit within the noise.