1
$\begingroup$

On which of theses two kinds of sample would a Random Forest (and more precisely sklearn RandomForest algorithm) give the best results ? (Y and other_features are continuous numerical variables, and the "cat_variable" modalities are not unbalanced).

 Y | other_features | cat_variable
...| ...            | A
...| ...            | B
...| ...            | B
...| ...            | B
...| ...            | C
...| ...            | A



 Y | other_features | is_A | is_B | is_C |
...| ...            | 1    | 0    | 0    |
...| ...            | 0    | 1    | 0    |
...| ...            | 0    | 1    | 0    |
...| ...            | 0    | 1    | 0    |
...| ...            | 0    | 0    | 1    |
...| ...            | 1    | 0    | 0    |

My question intend to be quite general : I'm looking for the best practice, and I want to know if binarizing categorical variables has an interest. If this is still unclear, I would be pleased if you explain me why.

$\endgroup$
  • $\begingroup$ NO, you should not $\endgroup$ – kjetil b halvorsen Sep 12 '18 at 20:36
  • 1
    $\begingroup$ RF is perfectly capable of handling categorical variables so there is no need for this. Also, you do not want to to create needless additional variables. $\endgroup$ – user2974951 Sep 13 '18 at 7:51
0
$\begingroup$

With increased sparsity one-hot-encoded features are less likely to be picked as splitting criterion compared to numeric values ('Y' and 'other_features').

Actually, you just need 2 bits to represent 'cat_variable': If is_A is 0 and is_B is 0, then of course is_C is 1. So you are introducing some sort of redundancy with 3 features.

Since RandomForests can handle categorical/nominal values, you can stick to 'cat_variable' as default.

But there is no general answer. You need to find out what works best for your dataset.

$\endgroup$
  • 1
    $\begingroup$ Could you explain why you say that there is no general answer ? All the arguments you presented are against the 2nd sample. Why do you finish with a nuanced conclusion ? $\endgroup$ – Doe Jowns Sep 13 '18 at 7:40
  • $\begingroup$ Edit: Replaced the 'So' by 'But'. $\endgroup$ – mlvalidated Sep 13 '18 at 7:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.