As I understand from this reference (cran.r-project.org/web/packages/rpart/vignettes/longintro.pdf , pp. 12-13) about the rpart package , the criteria for a making (or not) a new split in a decision tree is to compare the decrease in the error of the tree with the new split against the complexity parameter times the number of leaves it would yield. If the former is greater, then the split is made.

I've grown a tree of cp=0 for the whole iris dataset, and the tree I get is the following:

enter image description here

The predictor space is split as follows:

enter image description here

So my question is: if the complexity parameter is zero, why does the splitting end there? Why doesn't the algorithm split the versicolor node further with a Petal.Length<5.4? This would decrease the error of the tree, and since the complexity parameter is zero, it wouldn't prevent it from happening... or not??


Short answer: If you set minsplit=1 you may get what you want.

Long answer: CART build tree in 2 stages.

  • The first stage will grow tree as much as possible (with specified constrains)
  • The second stage will prune the tree use 1 SE rule

There are may parameters to control how to grow the tree and how to prune the tree, setting cp=0 is equal to say do not prune the tree. But we may need to set maxdepth, minsplit, minbuket etc. to control how to grow the tree.

This is my question and answer to the same question, with more complicated data than iris.

Why I cannot achieve 100% accuracy in my simple training data with CART model?

| cite | improve this answer | |
  • $\begingroup$ Wow, that was fast! =) And very clear. Thanks a lot. $\endgroup$ – Pablo Dec 14 '16 at 18:57
  • $\begingroup$ I'm still having doubts about this issue... When I train models with caret using method='rpart', most of the times the best fit is with cp=0, mainly because the tree is heavily pruned beforehand (caret doesn't split nodes with less than 20 instances) Therefore, I would like to grow a tree all the way down and then cross validate to find the best cp parameter, instead of adjusting the cp over a heavily pruned tree. Is that possible? I can do it manually with rpart, but it would be nice to use caret's built-in features to simplify the procedure. Any suggestions? $\endgroup$ – Pablo Jan 17 '17 at 11:38
  • $\begingroup$ @Pablo I think "most of the times the best fit is with cp=0" may not be accurate. The default parameter is cp=0.01, and the cross validation is conducted automatically. Please check function 'plotcp' and 'printcp'' $\endgroup$ – Haitao Du Jan 17 '17 at 14:09
  • $\begingroup$ Well, what I meant is that (at least for the models I've fitted) the best results are those with the smallest cp value. Obviously I don't think this will always be the case, but I feel that for these models a lower cp could improve the results, but that cannot be achieved unless the tree is more complex from the start. $\endgroup$ – Pablo Jan 17 '17 at 16:21
  • $\begingroup$ @Pablo I think you have problem of over-fitting, make sure your testing set is big enough. when you say "cp=0" seems to always better. $\endgroup$ – Haitao Du Jan 17 '17 at 16:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.