What are filter methods for feature selection?

Filter methods are feature-selection techniques that evaluate features using statistical properties of the data rather than repeatedly training the final machine learning model.

They are usually applied before model training.

How filter methods work

A filter method assigns each feature a score based on how useful or informative it appears to be.

Features with stronger scores are kept, while weaker ones may be removed.

Common filter methods

Examples include:

• correlation with the target
• mutual information
• chi-square tests for suitable categorical or count-based data
• ANOVA-style statistical tests
• variance thresholds

Each method measures a different type of relationship.

Example

Suppose a dataset contains 50 features.

A correlation-based filter may rank the numerical features according to how strongly they relate to the target.

The top-ranked features can then be selected for model training.

Why filter methods are useful

They are often:

• fast
• computationally inexpensive
• easy to apply to large datasets
• independent of a particular machine learning algorithm

Because they do not require training many candidate models, they are usually much faster than wrapper methods.

Main limitation

Many filter methods evaluate each feature individually.

This means they may miss situations where a feature is weak by itself but becomes useful when combined with another feature.

They may also keep multiple features that contain similar information unless redundancy is checked separately.

Filter methods are model-independent

A feature selected by a filter method is not chosen because it improved one specific model.

This can make the selected feature set reusable across different algorithms.

Avoid leakage

If a filter method uses the target, the feature scores must be calculated using training data only.

Running target-based feature selection on the complete dataset before validation can leak information from validation or test data.

In short, filter methods select features using statistical criteria before model training. They are fast and model-independent, but they may miss important feature interactions or redundancy between features.