I am new to decision trees, there are several things confused me a lot.

  1. The first thing is that should we convert all categorical variables (Such as: gender, department, number of years) to a numeric variables using unclass method?

  2. At the first time, I run the decision tree model simple using rpart(Y~.) based on the raw data which contains lots of ord. Factors, Factors,int. I took a look at the cptable for this model.

After I convert all factor variables into numeric using unclass, and build a new model based on the dataframe. I don't know why it can produce a totally different cptable. What I have done is just convert all factors to numeric, that is. Will it influence the model result? Why ?

  1. The last question is that if I am making a predictive model using regression trees, should I scale the data?

1 Answer 1

  1. You should convert any variables that should be numeric to numeric, e.g., number of years. It does not make sense to convert department to numeric. Any category that contains only two values (possibly gender, depending whether you work with two or more genders) can be left as is or changed to numeric, it does not make a difference to the tree, although it will be easier to interpret as categoric.
  2. Of course changing variables between categoric and numeric will change everything. If you leave number of years as categoric, then a tree will split it by assigning some numbers to the left child node and some to the right one - and probably using a wild mixture of assignments (0, 1, 2 to the left, 3 to the right, 4 and 5 to the left, 6 to the right...). If this variable is numeric, the tree will choose a threshold and assign every value below that threshold to the left and every number above the threshold to the right child.
  3. Scaling is possible, but not necessary. It will just change the threshold used in splitting. (If you have very large or small numbers, you may have numerical issues, but that is rare.)
  • 2
    $\begingroup$ And an overall note: the sample size required for this whole process to work is perhaps $n=100,000$. I mean 'work' in the sense of the patterns found by recursive partitioning and the accuracy of predictions being good in a new dataset. $\endgroup$ Sep 24, 2022 at 11:46
  • $\begingroup$ Thank you so much. It solved all my concerns, and may I ask are there any way to select the top split? From my perspective, it is impossible since the rpart will automatically arrange the splits from the top to the bottom. $\endgroup$ Sep 25, 2022 at 5:30
  • $\begingroup$ Most tree fitting algorithms will indeed do this automatically in order to optimize the fit, and you can't intervene. For the top split, this is however easily done: simply split your data into two however you want, and grow a separate tree for each subset. (I don't quite know why you would want to do this.) $\endgroup$ Sep 25, 2022 at 6:21
  • $\begingroup$ @StephanKolassa I tried to look into the tree_model$variable.important and found that variable1 and variable2 were the two most important variables since these two had the highest value in variable.important output. So my next step is try to build a new tree model with var1 and var2. $\endgroup$ Sep 25, 2022 at 8:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.