Jargonautica
Classical Machine Learning

Feature Engineering

Also called: Features

Deciding what to actually show the model. Raw data is rarely in a useful shape: a date of birth is less useful than an age, and separate columns for height and weight are less useful than the single number a doctor would calculate from them. For most traditional machine learning this is where the real work sits, and where most of the performance comes from.

Technical view

Covers transformation, scaling, encoding of categorical variables, binning, interaction terms and domain-informed aggregates. Deep learning reduced its importance by learning representations directly from raw input, which is the central claim of the field, but it remains essential for tabular data, where boosted trees plus good features still lead. The main hazard is target leakage through features that would not be available at prediction time, which produces excellent offline results and a worthless deployed model.


Part of AI Terminology Index, section: Classical Machine Learning.

How it connects

Feature Engineering links to 14 other terms, and the 12 closest are drawn here. The faint lines are links between those terms. See it in the full graph.

Data LeakageData LeakageRandom Forest and Gradient BoostingRandom Forest and G…Decision TreeDecision TreeDeep LearningDeep LearningClassification and RegressionClassification and …One-hot EncodingOne-hot EncodingLayersLayersLinear and Logistic RegressionLinear and Logistic…Feature ImportanceFeature ImportanceNormalisation and StandardisationNormalisation and S…Feature StoreFeature StoreMissing Data and ImputationMissing Data and Im…Feature Engineering