2
$\begingroup$

As I understand from this reference (cran.r-project.org/web/packages/rpart/vignettes/longintro.pdf , pp. 12-13) about the rpart package , the criteria for a making (or not) a new split in a decision tree is to compare the decrease in the error of the tree with the new split against the complexity parameter times the number of leaves it would yield. If the former is greater, then the split is made.

I've grown a tree of cp=0 for the whole iris dataset, and the tree I get is the following:

enter image description here

The predictor space is split as follows:

enter image description here

So my question is: if the complexity parameter is zero, why does the splitting end there? Why doesn't the algorithm split the versicolor node further with a Petal.Length<5.4? This would decrease the error of the tree, and since the complexity parameter is zero, it wouldn't prevent it from happening... or not??

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

Short answer: If you set minsplit=1 you may get what you want.

Long answer: CART build tree in 2 stages.

  • The first stage will grow tree as much as possible (with specified constrains)
  • The second stage will prune the tree use 1 SE rule

There are may parameters to control how to grow the tree and how to prune the tree, setting cp=0 is equal to say do not prune the tree. But we may need to set maxdepth, minsplit, minbuket etc. to control how to grow the tree.

This is my question and answer to the same question, with more complicated data than iris.

Why I cannot achieve 100% accuracy in my simple training data with CART model?

Improve this answer
$\endgroup$
5
  • $\begingroup$ Wow, that was fast! =) And very clear. Thanks a lot. $\endgroup$
    – Pablo
    Dec 14, 2016 at 18:57
  • $\begingroup$ I'm still having doubts about this issue... When I train models with caret using method='rpart', most of the times the best fit is with cp=0, mainly because the tree is heavily pruned beforehand (caret doesn't split nodes with less than 20 instances) Therefore, I would like to grow a tree all the way down and then cross validate to find the best cp parameter, instead of adjusting the cp over a heavily pruned tree. Is that possible? I can do it manually with rpart, but it would be nice to use caret's built-in features to simplify the procedure. Any suggestions? $\endgroup$
    – Pablo
    Jan 17, 2017 at 11:38
  • $\begingroup$ @Pablo I think "most of the times the best fit is with cp=0" may not be accurate. The default parameter is cp=0.01, and the cross validation is conducted automatically. Please check function 'plotcp' and 'printcp'' $\endgroup$
    – Haitao Du
    Jan 17, 2017 at 14:09
  • $\begingroup$ Well, what I meant is that (at least for the models I've fitted) the best results are those with the smallest cp value. Obviously I don't think this will always be the case, but I feel that for these models a lower cp could improve the results, but that cannot be achieved unless the tree is more complex from the start. $\endgroup$
    – Pablo
    Jan 17, 2017 at 16:21
  • $\begingroup$ @Pablo I think you have problem of over-fitting, make sure your testing set is big enough. when you say "cp=0" seems to always better. $\endgroup$
    – Haitao Du
    Jan 17, 2017 at 16:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.