How do you handle high-cardinality categorical features?

High-cardinality categorical features contain a large number of unique categories, such as thousands of product IDs, customer IDs, cities, or merchant codes.

They need special handling because simple one-hot encoding can create too many columns.

1. Group rare categories

Categories that appear only a few times can be combined into a common group such as:

Other

This reduces the number of unique values and can make the feature more stable.

2. Use frequency or count encoding

Each category can be replaced by how often it appears.

For example:

Category A → 1200
Category B → 85
Category C → 12

This keeps the feature compact, although it does not directly capture the relationship with the target.

3. Use target encoding carefully

Target encoding can replace each category with a statistic derived from the target.

It can work well for high-cardinality features, but it must be implemented using leakage-safe techniques such as out-of-fold encoding and smoothing.

4. Use hashing

Feature hashing maps many categories into a fixed number of numerical features.

This keeps memory usage under control even when the number of possible categories is very large.

The trade-off is that different categories can occasionally map to the same hashed representation.

5. Use models with native categorical support

Some algorithms can handle categorical variables directly or with specialized internal encoding.

Examples include implementations such as CatBoost and some configurations of LightGBM.

This can avoid creating thousands of one-hot columns manually.

6. Check whether the feature is actually useful

Some high-cardinality columns are identifiers rather than meaningful predictive features.

For example:

customer_id
transaction_id

may simply identify rows rather than provide reusable predictive information.

Such features should be evaluated carefully before encoding.

Handle unseen categories

Production data may contain categories that were not present during training.

The preprocessing pipeline should define a safe fallback for these unseen values.

Validate the encoding choice

Different encoding methods can change model performance, memory usage, and leakage risk.

The best choice should therefore be evaluated using proper validation.

In short, high-cardinality categorical features can be handled by grouping rare values, using frequency or target encoding, applying feature hashing, or choosing models with native categorical support, while carefully controlling leakage and unseen categories.