What is multicollinearity?

Multicollinearity occurs when two or more input features in a regression model are strongly linearly related to each other.

This means the features contain overlapping information, making it difficult for the model to determine the separate effect of each one.

Simple example

Suppose a dataset contains:

height_in_cm
height_in_meters

These two features contain almost exactly the same information.

If both are included in a linear regression model, the model has little basis for deciding how much effect should be assigned to each feature separately.

Another example

Imagine predicting house price using:

• total area
• number of rooms

These variables may be strongly related because larger houses usually have more rooms.

If several predictors move together strongly, multicollinearity can appear.

Perfect vs near multicollinearity

Perfect multicollinearity occurs when one feature can be calculated exactly from other features.

For example:

x3 = x1 + x2

Near multicollinearity occurs when features are not exact combinations but are still very strongly related.

Multicollinearity is mainly about predictors

The issue concerns relationships among the input features, not simply whether a feature is correlated with the target.

Two predictors can each be useful for predicting the target while still being highly correlated with each other.

It matters most for coefficient interpretation

Multicollinearity is especially important in models such as linear regression, where we often want to interpret the individual coefficient of each feature.

When predictors overlap heavily, separating their individual contributions becomes difficult.

How can it be detected?

Common approaches include:

• examining correlations between predictors
• checking whether features are derived from one another
• using diagnostic measures such as Variance Inflation Factor (VIF)

VIF is covered separately later in this section.

Not every correlation means severe multicollinearity

Some correlation between features is normal.

Multicollinearity becomes more concerning when predictors are strongly related enough that their separate effects become difficult to estimate reliably.

In short, multicollinearity occurs when predictor variables contain strongly overlapping linear information, making it difficult for a model—especially a regression model—to estimate their individual effects independently.