Jargonautica
How the Machinery Works

Deep Learning

Machine learning using neural networks with many layers. ‘Deep’ simply means lots of layers. This is the approach behind nearly every AI advance of the past fifteen years, from image recognition to speech transcription to chatbots.

Technical view

The underlying ideas date to the 1980s. What changed around 2012 was not theory but three practical things arriving together: enough labelled data, graphics processing units (‘GPUs’) that could do the enormous number of parallel multiplications required, and a handful of training refinements that made deep networks converge reliably. The field has since been driven substantially by scale. Empirical scaling laws show performance improving predictably as model size, data volume and computing power increase together, which is why the industry has invested so heavily in ever larger training runs.


Part of AI Terminology Index, section: How the Machinery Works.

How it connects

Deep Learning links to 14 other terms, and the 12 closest are drawn here. The faint lines are links between those terms. See it in the full graph.

Neural NetworkNeural NetworkLayersLayersFeature EngineeringFeature EngineeringConvolutional Neural NetworkConvolutional Neura…Artificial IntelligenceArtificial Intellig…Activation FunctionActivation FunctionComputer VisionComputer VisionMachine LearningMachine LearningRandom Forest and Gradient BoostingRandom Forest and G…Residual ConnectionResidual ConnectionGraphics Processing UnitGraphics Processing…Scaling LawsScaling LawsDeep Learning