What is Variance Inflation Factor (VIF)?

Variance Inflation Factor (VIF) measures how much the variance of a regression coefficient is increased because a predictor is linearly related to the other predictors.

It is commonly used to diagnose multicollinearity in regression models.

How VIF is calculated

For a feature Xj, first predict Xj using all the other input features and calculate the resulting R².

Then:

VIF = 1 / (1 - R²)

If the other predictors explain a large portion of Xj, the R² is high and the VIF becomes large.

Simple example

If:

R² = 0.80

then:

VIF = 1 / (1 - 0.80)
    = 5

This means the variance of that coefficient is inflated relative to a situation where the predictor is not linearly related to the other predictors.

How VIF is interpreted

Common rules of thumb are:

• VIF = 1: no linear relationship with the other predictors
• VIF between 1 and 5: some correlation, often not severe
• VIF above 5: worth investigating
• VIF above 10: often treated as a strong multicollinearity warning

These are guidelines rather than universal cutoffs. The acceptable level depends on the modeling goal and domain.

Why a high VIF matters

A high VIF can indicate that a regression coefficient may be:

• unstable
• sensitive to small changes in the data
• associated with a larger standard error
• difficult to interpret independently

High VIF does not automatically mean remove the feature

A feature with a high VIF may still contain useful information.

Before removing it, consider:

• whether another feature contains nearly the same information
• which feature is easier to interpret
• whether prediction or coefficient interpretation is the main goal
• whether regularization could help

VIF mainly diagnoses linear dependence among predictors

VIF is designed around linear relationships between input variables.

It does not detect every possible nonlinear dependency between features.

In short, VIF measures how strongly multicollinearity inflates the uncertainty of a regression coefficient. A larger VIF indicates that the predictor overlaps more strongly with the other predictors and may be harder to interpret reliably.