1
$\begingroup$

I'm faced with a data set where one of the features is a set of 4-5 categories (this number of categories isn't constant). I need to use this feature for building a decision tree. I searched online for any clustering techniques where this could work, to no avail.

One approach could be to use columns for each of the categories and set 1 or 0 depending on whether that category appears in the set or not, but since the total number of categories is very large ( ~ 2000) it is not a viable option.

Moreover, I'm interested in finding the rules, more so than building an accurate model. So the final rules obtained from the decision tree should be such that we can interpret them in terms of the categories.

Any help would be appreciated.

$\endgroup$
4
  • $\begingroup$ Are you saying that your data are categorical, not quantitative? $\endgroup$
    – ttnphns
    Jan 16 '15 at 8:55
  • $\begingroup$ @ttnphns While most of the features are numeric/quantitative, this particular one, in question is a set of categories. $\endgroup$
    – wrahool
    Jan 16 '15 at 9:00
  • $\begingroup$ You shouldn't use K-means if some features are categorical. $\endgroup$
    – ttnphns
    Jan 16 '15 at 9:04
  • $\begingroup$ Apologies, I meant decision tree. Not clustering. I've edited the question accordingly. $\endgroup$
    – wrahool
    Jan 16 '15 at 9:10
1
$\begingroup$

You could build a feature like "rarity" and use that. What I mean is that you calculate frequency table for the categories - how many times each feature is present in the dataset (and divide it by total number of features present in dataset to make it a percentage).

Then you can add together (or take a mean or median or sd or all of them) these percentages for each observation, based on the categories that are present for that observation. It would result in a variable between 0 and 1, where smaller values indicate that the categories are rare in the dataset and higher values indicate that these are common categories.

$\endgroup$
2
  • $\begingroup$ But how would you interpret the rules? The rule will yield an aggregate score which is combination of the values corresponding to each of the categories, but there will be no way of tracing the categories back, from the score. $\endgroup$
    – wrahool
    Jan 16 '15 at 9:45
  • $\begingroup$ I made the assumption that the categories are not distributed uniformly, that some are way more common than others and that would help. But you are right, it wouldn't link directly back to categories. Knowing a bit more about the problem domain and the categories would be helpful :) $\endgroup$
    – LauriK
    Jan 16 '15 at 9:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.