What is the difference between feature engineering and feature extraction?

Feature engineering and feature extraction both improve how data is represented for a machine learning model, but they are not exactly the same.

Feature engineering

Feature engineering is the broader process of creating, transforming, combining, or selecting features to better represent the prediction problem.

Examples include:

• creating age from date_of_birth
• calculating debt_to_income_ratio
• extracting day_of_week from a timestamp
• grouping rare categories
• applying a logarithmic transformation

It often uses domain knowledge to design features that make important relationships easier for the model to learn.

Feature extraction

Feature extraction transforms existing raw data into a new representation that captures useful information.

Examples include:

• using PCA to convert many correlated variables into principal components
• converting text into numerical representations
• extracting statistical summaries from sensor signals
• converting raw image information into more compact feature representations

The extracted features may be less directly interpretable than the original inputs.

Key differences

• Scope: Feature engineering is the broader concept. Feature extraction is one way of creating a useful representation from existing data.
• Typical goal: Feature engineering often introduces domain-informed features, while feature extraction often compresses or transforms raw data into informative components.
• Interpretability: Engineered features such as age or price_per_unit are often easy to understand. Extracted features such as PCA components may be harder to interpret.
• Dimensionality: Feature engineering can increase, reduce, or keep the same number of features. Feature extraction often reduces or reorganizes the original feature space.
• Method: Feature engineering may involve manual rules and domain knowledge, while feature extraction is often performed using systematic statistical or algorithmic transformations.

They can be used together

A machine learning pipeline may use both.

For example:

Raw data
→ Create domain-specific features
→ Apply feature extraction
→ Train model

Both must respect the training boundary

If either process learns information from the dataset, it should be fitted using training data only and then applied consistently to validation and test data.

In short, feature engineering is the broader process of designing useful model inputs, while feature extraction specifically transforms existing data into a new, often more compact or informative, representation.