How do you detect outliers?

Outliers can be detected by looking for observations that are unusually far from the rest of the data.

The best method depends on the feature distribution, number of variables, and the meaning of the data.

Start with domain checks

Some outliers can be found using known valid ranges.

For example:

• an age of 250 years
• a negative product price
• a temperature outside the possible range of the sensor

These may indicate data-entry or measurement errors.

Use visual inspection

Simple plots can reveal unusual values quickly.

Useful visualizations include:

• box plots
• histograms
• scatter plots

A point far away from the main cluster may be an outlier worth investigating.

Use the IQR method

The interquartile range (IQR) is a common robust method for numerical data.

First calculate:

IQR = Q3 - Q1

A common rule marks values outside these limits as potential outliers:

Lower bound = Q1 - 1.5 × IQR
Upper bound = Q3 + 1.5 × IQR

Values beyond these bounds are candidates for investigation.

Use z-scores

For data that is reasonably close to a normal distribution, a z-score measures how many standard deviations a value is from the mean.

Very large positive or negative z-scores can indicate possible outliers.

A threshold such as |z| > 3 is sometimes used as a practical rule, but it should not be treated as universal.

Detect multivariate outliers

Some records may look normal in each feature separately but become unusual when several features are considered together.

Methods such as:

• Isolation Forest
• Local Outlier Factor
• clustering-based approaches

can help detect these more complex patterns.

Investigate before acting

An outlier flag means the record is unusual, not necessarily wrong.

After detection, ask:

• Is the value valid?
• Is it a data error?
• Is it an important rare case?
• Does it belong to the same population as the rest of the dataset?

In short, outliers can be detected using domain rules, visualizations, statistical methods such as IQR and z-scores, or multivariate techniques. Detection should always be followed by investigation before deciding what to do with the record.