Jargonautica
Classical Machine Learning

Pseudo-labelling and Self-training

Also called: Pseudo-labelling, Self-training

Using a model’s own confident predictions as training data for the next round. It can work well and it can also quietly amplify the model’s existing mistakes, since nothing in the loop distinguishes confident and right from confident and wrong.

Technical view

Iteratively label unlabelled data above a confidence threshold, retrain, repeat. Confirmation bias is the central risk, mitigated by threshold tuning, ensembling, augmentation-based consistency, and injecting noise into the student as in noisy student training. It is directly analogous to distillation with a self-teacher, and to synthetic data generation for language models, where the same amplification concern applies.

Part of AI Terminology Index, section: Classical Machine Learning.

How it connects

Pseudo-labelling and Self-training links to 4 other terms. The faint lines are links between those terms. See it in the full graph.

Synthetic DataSynthetic DataDistillationDistillationModel Confidence and CalibrationModel Confidence an…Semi-supervised LearningSemi-supervised Lea…Pseudo-labelling and Self-training