What is train-test contamination?

Train-test contamination happens when information from the test dataset influences the training or model-development process.

This breaks the independence of the test set and can make the final evaluation look better than the model will perform on truly unseen data.

Simple example

Suppose the same customer record, or a nearly identical duplicate, appears in both the training set and the test set.

The model may effectively see that example during training and then be evaluated on it again.

That makes the test result less trustworthy.

Common causes of train-test contamination

Contamination can happen when:

• duplicate or related records appear in both training and test sets
• preprocessing is fitted on the full dataset before splitting
• feature selection uses information from the test set
• imputation or scaling statistics are calculated using test data
• the test set is repeatedly used to choose models or tune hyperparameters

Why contamination is a problem

The purpose of the test set is to simulate data the model has never seen before.

If test information influences training, the evaluation is no longer independent.

This can lead to:

• inflated test scores
• hidden overfitting
• incorrect model comparisons
• unrealistic expectations for production performance

Example with preprocessing

Suppose missing values are filled using the mean calculated from the entire dataset before creating the train-test split.

The training process has now indirectly used information from the test set.

A safer approach is to calculate the mean using only the training data and then apply that learned value to the test data.

Test data should remain isolated

The test set should be treated as unseen data until the final evaluation.

Training, preprocessing, feature selection, and model tuning should be completed without using test-set information.

In short, train-test contamination occurs when information from the test set leaks into training or model development, causing the test score to overestimate real-world performance.