0
$\begingroup$

I am currently reading the book "Random Forests" by Yu. L. Pavlov. Then it came across my mind the question If I were to use ensembled tree, say XGBOOST, do I need to transform each categorical variable into a dummy variable.

I have searched the internet and found mixed results People saying that dummy variables impacts performance: article 1, article 2.

but majority of the articles I have read agrees with the notion that categorical variables needs to be encoded.

I would like to hear your thoughts on this, and if you can also point to any journal article tackling this matter, that would be a very huge help.

$\endgroup$
0
$\begingroup$

I would dummy it up but it kind of depends, for example years of education and predicting income. You can and should dummy it in a normal linear regression but for a tree it makes sense to leave it because the tree can learn the highschool/bachelor/master/phd splits which are more important than average incremental differences between each year. So you are safer to use un-dummied data if there is that underlying relationship. But if there is no underlying relationship and for example they are just ids like product 3, 4, 5 it will bucket them as if there is a relationship OR the tree will spend a lot of it's splits just splitting on the product id anyway so obviously you are better off creating dummies. So I would say it is entirely problem/variable dependent. And for boosted trees lightgbm typically does a lot better with loads of sparse dummies in terms of both speed and accuracy over xgboost, so just try dummies and hand it off to lightgbm and iterate from there.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.