I have a dataset listing the software installed for each user. This dataset shall be used (in conjuction with other user datasets) to classify the user into 4 (imbalanced) categories.

There are over 1000 different pieces of software. Each user can have between 15 and 40 pieces of software installed. In fact, the same software can be counted as installed multiple times (I presume this refers to different versions/upgrades of the same software but I'm not sure).

The naive approach (A) would be to use one-hot/dummy encoding for a variable "Software" but this would produce over 1000 features.

A slightly more sophisticated approach would use some sort of feature selection (e.g. using chi-square/mutual information or some wrapper model) and ignore the less informative pieces of software.

And a third one would be to use target/mean encoding.

Which of these would you recommend?

My preference is for target/mean encoding but I wonder how to approach it.

I was thinking initially to just sum up the raw counts for each target value: I'd create 4 features each one corresponding to one of the values of the dependent variable and for each of these I'd count the times the particular software was installed and then sum up over all software installed by the user. The problem is that in this way some popular software (e.g. Word or a browser) will likely "crowd-out" some less commonly used (but potentially more informative) pieces of software. So next I thought that I could replace the raw counts with the percentage. This would give equal weight to each piece of software. But is this OK?

I guess I am probably not the first person to face such a problem. What would constitute a principled/established approach of dealing with such variable number of high cardinality categorical features?

  • $\begingroup$ You could group all software into N groups and then use the number of apps within a group as a feature. I recommend using domain knowledge. $\endgroup$ – Nikolas Rieble Nov 28 '18 at 14:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.