What is data leakage in machine learning?

Data leakage in machine learning happens when information that should not be available to the model influences the training process or model evaluation.

It can make a model appear much more accurate during development than it will be in real-world use.

Simple example

Suppose a model is built to predict whether a customer will cancel a subscription.

If one input feature records whether the customer later submitted a cancellation request, the model is indirectly being given information about the outcome it is supposed to predict.

That is leakage because the information would not have been available at the actual prediction time.

Common ways leakage happens

Data leakage can occur when:

• future information is included in input features
• information closely related to the target is accidentally used as a feature
• test or validation data influences training
• preprocessing is fitted using the full dataset before splitting
• duplicate or related records appear across training and evaluation sets

Why leakage is dangerous

A leaked model may show excellent validation or test metrics because it has access to information that makes the task artificially easy.

After deployment, that information may no longer be available, so model performance can drop sharply.

Leakage can therefore lead to:

• overly optimistic performance estimates
• poor generalization
• incorrect model selection
• false confidence in deployment readiness

Leakage can be subtle

Some leakage is obvious, such as directly including the answer in the input.

Other cases are harder to notice, such as using statistics calculated from future observations or allowing related records to appear in both training and test sets.

In short, data leakage occurs when information that would not legitimately be available at prediction time or during independent evaluation enters the machine learning process, causing misleadingly strong results.