# Reducing number of levels of unordered categorical predictor variable

I want to train a classifier, say SVM, or random forest, or any other classifier. One of the features in the dataset is a categorical variable with 1000 levels. What is the best way to reduce the number of levels in this variable. In R there is a function called combine.levels() in the Hmisc package, which combines infrequent levels, but I was looking for other suggestions.

• Is the categorical variable unordered? Approximately how many cases do you have? What is the frequency distribution across the categorical variable? – Jeromy Anglim May 6 '11 at 3:32
• The levels are not ordered. I have around 10,000 observations. The frequency distribution is as follows: level A appears in around 11% of the observations. Level B appears in 8%. Level c appears in 5%. About 15 of these levels cover 50% of the observations in the dataset. – sabunime May 6 '11 at 14:56

If the levels are not ordinal you can cluster the levels based on other features/variables in your dataset and substitute the cluster ids for the previous levels. There are as many ways to do this as there are clustering algorithms, so the field is wide open. As I read it, this is what combine.levels() is doing. You could do similarly using kmeans() or prcomp(). (You could/should subsequently train a classifier to predict the clusters for new datapoints.)