For example a column containing numeric values for phone area code or a postal code.
In case it matters, I am preprocessing data for use in a tree-based ensemble classifier.
3
-
$\begingroup$ You might get contradictory answers to this question, because a close version of it has been controversial. See psycnet.apa.org/record/2005-07821-008. $\endgroup$– whuber ♦Commented Nov 1, 2021 at 21:32
-
$\begingroup$ I guess I am interested in common practices from people here. To me, it makes more intuitive sense for a tree to make a split on a numeric representation of the categorical variable where I control the encoding method. But I don't know if I'm wasting my time doing this. $\endgroup$– gsm113Commented Nov 1, 2021 at 21:37
-
2$\begingroup$ The whole point of the paper I referenced is that sometimes the "common practice" is the inferior one. Useful information can be lurking in the numerical representations of variables that appear to be purely nominal. $\endgroup$– whuber ♦Commented Nov 1, 2021 at 21:42
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Yes, it is important, otherwise, a model cannot correctly learn effects for individual factors. However, some tree-based implementations do not do that because it is computationally more efficient to have one variable with 100 numeric values than to have 100 dummy variables, and a tree will eventually split all levels anyway, so the arbitrary ordering is not such a big issue as it is in other models, like linear regression.
If there is information "accidentally" encoded in the numeric code, then I would argue that you should still treat them as categorical variables and instead put that information explicitly in your model. So if a zip code is roughly related to geographic location, then it would be better to put the categorical zipcode in location to your model explicitly than treating zipcode as a numeric feature and hoping a model will figure it out. Also, if there is some information encoded in the numerical values, you should be extra cautious because this might lead to data leakage problems
