I am trying to grow a classification tree with a few continuous explanatory variables and a few factor variable.

It seems the Rpart alogrithm is ignoring the factor variables. The differences are significant among the different factor levels,but there is no node associated with any of the factor variables. To make sure that it was not feature of the data set in ran the following simulation.

There was one continuous variable -Income which was random normal with mean 0 and sd 0.25. The other variable was a sector variable that was either U or R with probability 0.5.

Finally the probability of Yes was defined as

If Sector==U, then pr<-1/(1+exp(-income+1.0) If Sector==R,then pr<-1/(1+exp(-income*2 +1.5)

Hence i should expect that classification tree picks out effect of sector. But what i am getting is a lot of splits on income. Is it a general problem?

  • $\begingroup$ Try setting the cp parameter to a smaller value. $\endgroup$ – Hong Ooi Jul 16 '15 at 15:17

I would sugget, following Hong Ooi's comment, to experiment and play with the control parameter in rpart. and particularly the minsplit and the cp values. (e.g. in R, control=rpart.control(minsplit=50, cp=0.005), etc. ). That should hopefully show some different trees, and will give intuition why certain variables are not selected. I would recommend using the automatic cross-validation in the rpart package to select the cross-validated cp value.

  • $\begingroup$ Thank you. I reduced cp to 0.00001 and got the tree to pick up locality. But the model is still have a very high error rate with a lot of false negatives. Is there any literlature on how to build trees when the number of success /positive outcomes is small in comparison with negative outcomes? Thanks $\endgroup$ – kangkan Dc Jul 17 '15 at 19:01
  • $\begingroup$ rpart allows you to specify a cost-matrix that you can use to adjust the costs of a false positive vs. the cost of a false negative. see the rpart documentation. also - in e.g. caret package - you can downsample the data to adjust for the class imbalance $\endgroup$ – Wouter Jul 18 '15 at 10:23
  • $\begingroup$ thank you.I got much better results with using a different loss matrix. I also used the SMOTED data, and got similar results. I was wondering if there is a way to shift the probablity threshold downwards from 0.5 while making the classification predictions. ? $\endgroup$ – kangkan Dc Jul 18 '15 at 18:08
  • $\begingroup$ you can have the predict funciton output probabilities - and then threshold them yourself $\endgroup$ – Wouter Jul 19 '15 at 11:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.