I overheard a colleague discussing her strategy for using categorical features the other day, and she mentioned that instead of one-hot-encoding, she does something like this:
cat_string cat_num dog 0 cat 1 dog 0 dog 0 horse 2
Then she feeds
cat_num (along with other inputs, potentially) into her model(s), without one-hot-encoding.
Now, there's obviously no issue if she wanted to assign these numbers to the categories and one-hot-encode - inputs created by OHEing
cat_num would be identical, and she would actually be doing a sort of file compression if she dropped
cat_string and just preserved that mapping - so nothing bad about that.
I could see not one-hot-encoding
cat_num maybe being fine in two other cases
1) Where there is some natural ordering to the categories, e.g.
high, and that ordering is reflected in the integer encoding applied, e.g.
medium: 1, and
cat_num has relatively low cardinality, no inherent ordering, and you're using a tree-based models - the idea there being that, with a well-specified tree, your tree might identify reasonable split points and effectively "learn" the one-hot-encoding, in a way. But that seems to be poor practice, when one-hot-encoding looks to be the best-practice way of doing things. OHEing, IMO and in general, makes things much easier to understand for these types of scenarios, and is trivial if you have enough computing power. I don't think this second approach would scale to high-cardinality categorical features with no inherent ordering, as learning "important" splits like
cat_num <= 1000 seems like nonsense. Especially with a linear model, measuring the effect on the output from a "one-unit increase in
cat_num", holding all else constant, is nonsense.
Can anyone else offer insight to this question?