0
$\begingroup$

When a variable is discretized it is converted to a categorical variable. This new variable should be encoded back to numeric using label encoding or one hot encoding. I mean, sklearn by default (https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.KBinsDiscretizer.html) uses one hot encoding but maybe it makes more sense to use label encoding. However, if I convert it back to numeric using Label Encode I don't really see the point of discretization. What's the best default: one hot encoding or label encoding?

As you can see it's not a specific question but general question about best defaults.

$\endgroup$

1 Answer 1

0
$\begingroup$

Depends on the algorithm!

Linear/additive models and NNs work quite well with OHE.
Tree based methods don't, so you are better off using binary or numerical (label) encodings.
Finally, you can also use frequency encoding or some type of target encoding (mean, for example), which often offer good results independently from the model.

However, I don't really see why you would discretize the variable in the first place if it's continuous, except in some particular cases for performance...

EDIT: To be more precise on my statement about trees, I reference the original CART paper by Breiman [1984]. Finding the optimal split for a categorical variable with L categories would mean to evaluating $2^L-1$ different splits. Breiman proves that the optimal split can be found by ordering categories by mean response (or class1 probability in binary classification problems) and only evaluating the L splits of the ordered categories.

This is all and all equivalent to a target encoding, and it is proven to be the best approach for regression and binary classification. For multiclass classification the problem is more complicated.

$\endgroup$
3
  • $\begingroup$ You say that OHE is not recomended for trees but I have seen most people use it for XGBoost. $\endgroup$
    – vbn
    Dec 13, 2019 at 10:24
  • $\begingroup$ Here's a very nice post on the subject: medium.com/data-design/… -- OHE is just not recommended. The implementations that allow for categorical variables for regression and binary classificaiton usually implement a sort of target encoding, assigning a number to the categories based on their average regression value/probability of being in the positive class (I don't remember which paper this was in, I'll post it if I find it) $\endgroup$
    – Davide ND
    Dec 13, 2019 at 10:40
  • $\begingroup$ And this: roamanalytics.com/2016/10/28/… -- in general - OHE is easy to use and not too bad as long as cardinality is small :) $\endgroup$
    – Davide ND
    Dec 13, 2019 at 10:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.