What are outliers in machine learning data?

Outliers are data points that are unusually far from most other observations in a dataset.

They may represent rare but valid cases, measurement errors, data-entry mistakes, or unusual events.

Simple example

Suppose a feature contains these monthly incomes:

35,000
42,000
39,000
41,000
2,500,000

The value 2,500,000 is very different from the others and may be considered an outlier.

Why outliers occur

Outliers can appear because of:

• genuine rare events
• measurement or sensor errors
• data-entry mistakes
• incorrect units
• unusual customer or system behavior
• natural variation in the population

An outlier is not automatically bad data.

Why outliers matter in machine learning

Some algorithms are sensitive to extreme values.

Outliers can:

• pull the mean away from typical values
• affect scaling
• increase prediction error
• influence regression lines strongly
• make some distance-based methods behave poorly

Other models may be more robust to outliers.

Outliers can also be important

Sometimes the unusual cases are exactly what the model needs to learn.

For example:

• fraudulent transactions
• equipment failures
• unusual medical cases
• abnormal network activity

Removing these observations could remove valuable information.

Investigate before changing them

When an outlier is found, ask:

• Is the value possible?
• Was it recorded correctly?
• Does it belong to the same population as the rest of the data?
• Is it important to the prediction problem?

Depending on the answer, an outlier may be kept, corrected, transformed, capped, or removed.

In short, outliers are unusually extreme observations. They can distort some machine learning models, but they may also represent important real-world cases, so they should be investigated rather than automatically deleted.