What is standardization in data preprocessing?

Standardization is a data preprocessing technique that transforms a numerical feature so its values are expressed relative to the feature's mean and standard deviation.

It is commonly used to put features with different units or scales onto a more comparable basis.

Standardization formula

A common formula is:

z = (x - mean) / standard_deviation

where:

• x is the original value
• mean is the average value of the feature
• standard_deviation measures how spread out the feature values are

After standardization, the transformed feature typically has a mean close to 0 and a standard deviation close to 1.

Simple example

Suppose a value is:

x = 70
mean = 50
standard deviation = 10

Then:

z = (70 - 50) / 10 = 2

The standardized value is 2, meaning the original value is two standard deviations above the mean.

Why standardization is useful

Standardization can help algorithms that are sensitive to feature magnitude, distances, gradients, or coefficient sizes.

It allows features measured in very different units to contribute on a more comparable scale.

Standardization does not force values into a fixed range

Unlike min-max normalization, standardized values are not limited to a range such as 0 to 1.

Values can be negative, positive, or greater than 1 in magnitude.

Outliers can still affect standardization

Because standardization uses the mean and standard deviation, extreme values can influence both statistics.

For heavily skewed data or strong outliers, a more robust scaling approach may sometimes be preferable.

Fit standardization on training data only

The mean and standard deviation should be calculated from the training set.

The same learned values must then be used to transform validation, test, and future production data.

This prevents evaluation data from leaking into preprocessing.

In short, standardization rescales a numerical feature by subtracting its mean and dividing by its standard deviation, producing values that are easier for many scale-sensitive machine learning algorithms to work with.