Categorical Variable with too many levels for Decision Tree I am trying to build a decision tree but the problem is I have too many levels on one of my categorical variable. The variable is 'source' - It indicates the source website where the user came from. I want to include this variable in my decision tree. How to deal with the many levels?
 A: I can think of three strategies:

*

*Use a random effects model. This shrinks the predicted outcome for low frequency categories towards the grand mean and is usually the best approach.

*Group all low-frequency categories into "other"

*Classify all low-frequency URLs by category and use the categories rather than the URLs; leave higher-frequencies categories as they are

A: One tactic is finding quantitative characterizations, or at the very least categorical characterizations with fewer levels, of the original levels. The most obvious is mean encoding, but you can try out anything you can think of: length of URL, top level domain name, etc. Take a look at the URLs and see if any categorization occurs to you.
Another tactic is ensemble modelling. Reducing the number of features can increase the number of levels that a model can usefully incorporate, so if you can find subsets of the features that yield useful results, you can build a model for each subset, and then build an ensemble model.
Keep in mind that the more things you throw at it, the higher the risk of overfitting.
A: To get rid of high cardinality features, you have options

*

*Frequency encoding, which encodes the frequencies of the entities instead of their categorical values

*Mean encoding (beware of possibility of overfitting) because you'll be using target information

*Hashing

*and some others.

As a first step, I'd go with frequency encoding, it's simple and less risky compared to mean encoding, and more meaningful than hashing. To calculate website frequencies, use your training data (not testing). You may need to preprocess your data to accommodate for Other Website category, since you may have new websites in the test data.
You may also want to use existing architectures like doc2vec, or train separate autoencoders for the websites, but I'd first start simple.
