1
$\begingroup$

I have a dataset with several categorical columns which I was planning to flatten into binary categories.

Let's say I have three features in my dataset.

  • Feature1: has got 300 different values (numeric values)
  • Feature2: has got 450 different values (string values)
  • Feature3: has got 900 different values (string values)

Now instead of having a three with three nodes, one per feature, I decided to flatten this three features into binary features. So at the end I will have 300+450+900 = 1650 binary features where only 3 will have a value 1, and the other 1647 will have a zero.

I am not sure if it is the best approach, so I was considering to bin the different options into groups of 10 options of the same feature into the same bin. So at the end I will have just 165 binary features which will represent the original 1650.

We the binning I will be loosing information, but I the model size will be smaller.

I don't have much experience with Random Forest and I am not sure how many features it can handle without problem, and from what number I should start considering the binning approach.

I also would like to mention that I am planning to use RandomForest with the Spark-scala API. (Just in case it may be relevant for the question)

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

It is probably not a good idea, unless thou somehow know that your binary features are fantastic features that RF would otherwise not figure out due to limited data. You are throwing away information of the more detailed scale and RF is less goos at handling variables with few cut points - I suspect giving it the full continuous scale is likely much easier for it. You could of course compare and see. RF as an algorithm has no inherent limitation on features, but as if any algorithm lots of truly useless features will not help and likely degrade the performance.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks for the answer. Another possible approach I didnt mention buy may suit what you recommend is look for the long tail, and those features which has got a really small number of ocurrences join then in a category named others. I believe this is a less agresive binning option. Please let me know what you think about this last approach $\endgroup$
    – Ignacio Alorre
    Commented Sep 10, 2018 at 6:59
  • 1
    $\begingroup$ Perhaps with the string values (but not with the numeric scale), if some categories are really tiny and somehow similar it may be a reasonable idea. Once you have a decent number of cases for each category, I would not. $\endgroup$
    – Björn
    Commented Sep 10, 2018 at 7:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.