Jargonautica
The Foundations

Transfer Learning

Reusing what a model learned on one job to help it on another. A system trained on millions of general photographs already knows about edges, textures and shapes, so teaching it to spot a specific manufacturing defect takes a few hundred examples rather than a few million. It is the reason most organisations never train anything from scratch.

Technical view

Representations learned on a large general task transfer because early and middle layers capture features that are broadly useful rather than task-specific. In practice you take a pre-trained model and either freeze most of it and retrain only the final layers, or fine-tune the whole thing at a low learning rate. Transfer works best when the source and target domains are related; when they are not it can actively hurt, a failure sometimes called negative transfer. Nearly all modern practice, including every use of a foundation model, is transfer learning under another name.


Part of AI Terminology Index, section: The Foundations.

How it connects

Transfer Learning links to 6 other terms. The faint lines are links between those terms. See it in the full graph.

Fine-tuningFine-tuningCatastrophic ForgettingCatastrophic Forget…Pre-trained ModelPre-trained ModelParameter-efficient Fine-tuningParameter-efficient…Foundation ModelFoundation ModelDeep LearningDeep LearningTransfer Learning