1
$\begingroup$

I have two features in my dataset I'm using to help predict a binary outcome. Based on my features, I'm trying to figure out which I need to drop a dummy to avoid the dummy trap.

  1. First feature is a string of "tags". There are 239 unique tags, which are seen in the dataset as comma separated sets of 1 to 9 tags.

    • What I did here was split them by the delimiter into a vector of 1s or 0s using MultiLabelBinarizer(). So these will be represented as a vector of length 239 of 1s and 0s.

  2. Second feature is country codes, eg. (US, MX, FR, etc.). There are 26 unique country codes.

    • These are represented as a single country for each row of the dataset. I plan on one hot encoding these as well to 0s or 1s.

I'm fairly sure I don't need to drop anything for the first feature. It's so sparse, that I can't imagine it requiring that.

Where I'm stuck though is about the country code, I can't understand if I need to drop one column or if I can just keep them all.

If I don't drop, in the end I will have 26 + 239 columns of ones and zeros, mostly zeros. If I do drop one, I will have 25 + 239 of ones and zeros.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

do the usual one hot encoding you are mentioning, then you can check if the sum of the (ones and zeros) of your new vector add to a constant for all your observations. If so, then you can eliminate one categorical variable as its not independent. For example: encoding Men/Woman for Gender attribute of persons would always sum to one, in this case you can remove one (Man or Woman), thats the textbook example of dummy variable trap.

If you have some insights on your data, you can also see directly if you have the dummy trap (does each observation in your data is assigned to only one country?, and, Do all observations have one country assigned?, if yes to both, then you should remove one country when encoding).

If you have observations with missing values for the categorical features you would need to think a bit how to implement things. If its Gender it is clear that missing values should either be male or female, but for countries might be different. And this would impact the dummy variable thing.

cheers

Improve this answer
$\endgroup$
4
  • $\begingroup$ Yes, the sum of the new vector adds to a constant of 26 for all the observations for this particular feature. So for this feature I will drop a country. For the other feature I'm assuming I do not need to drop anything since the sum of the vectors are not equal to a constant for all the observations. Correct assumption? $\endgroup$
    – Kevin
    Feb 4, 2019 at 18:32
  • $\begingroup$ More I think about it, sum of the first feature <> constant, sum of the second feature == constant. But sum of feat_1 and feat_2 <> constant. So does that matter? $\endgroup$
    – Kevin
    Feb 4, 2019 at 18:36
  • $\begingroup$ hi, then i think is ok to drop one country, as having zeros in all the others means it should be the one you removed. You ask "But sum of feat_1 and feat_2 <> constant. So does that matter?" I dont think so, i am guessing <> means "not equal", this is just because the sum of feature 1 is not constant as you mention. But still feature 2 has the dummy variable trap if you dont remove one country as all 26 are not independent. $\endgroup$
    – Alejandro Celis
    Feb 4, 2019 at 20:01
  • $\begingroup$ Okay. I did drop, but it's interesting the results didn't change in a meaningful way. $\endgroup$
    – Kevin
    Feb 4, 2019 at 20:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.