8
$\begingroup$

As the complexity parameter is calculated? What is the meaning of it?

From what I read, the cp is a value at which the tree makes divisions in the nodes until the reduction in the relative error is less than a certain value.

There are places I read that say the CP affects only the growth of the tree and others say that interferes with pruning too. For min appears that it interferes only in growth but not sure.

I am using rpart () package to create trees, in the case of the classification tree exists missclassification rate to evaluate the ratings, but in the case of regression is not anything to evaluate the predictions beyond the MSE?

Improve this question
$\endgroup$
2
  • $\begingroup$ Have a look at this stats.stackexchange.com/questions/117908/… $\endgroup$
    – Simone
    Commented Oct 31, 2015 at 13:58
  • $\begingroup$ @Simone I've read it, but I did not understand well if it goes for regression tree also $\endgroup$
    – user72621
    Commented Oct 31, 2015 at 14:09

1 Answer 1

Reset to default
3
+50
$\begingroup$

This is answered in this rpart resource. From p. 25:

For regression models (see next section) the scaled cp has a very direct interpretation: if any split does not increase the overall $R^2$ of the model by at least cp (where $R^2$ is the usual linear-models definition) then that split is decreed to be, a priori, not worth pursuing. The program does not split said branch any further, and saves considerable computational effort.

That same page gives this formula for how the cp parameter affects calculation of a tree's risk:

$$R_{cp}(T) ≡ R(T) + cp ∗ |T| ∗ R(T_1)$$

($T_1$ here is a tree with no splits, $|T|$ the splits in the tree. The full formal definition of risk is outside the scope of your question, but for reference the definition is on p. 4.)

Improve this answer
$\endgroup$
6
  • $\begingroup$ But the error associated with the cp would be cross-validation error? $\endgroup$
    – user72621
    Commented Nov 2, 2015 at 19:58
  • $\begingroup$ I'm not sure what you mean by "the error associated with the cp," but based on this quote from the same page, cp doesn't appear to directly involve CV error: "The default value of .01 has been reasonably successful at ‘pre-pruning’ trees so that the cross-validation step need only remove 1 or 2 layers, but it sometimes over prunes, particularly for large data sets." More, section 4.2 mentions setting the complexity parameter via CV. $\endgroup$
    – Sean Easter
    Commented Nov 2, 2015 at 21:57
  • $\begingroup$ The $cp$ is involved both in the growth of the tree as in pruning then? $\endgroup$
    – user72621
    Commented Nov 2, 2015 at 22:35
  • $\begingroup$ It doesn't appear that way to me. It seems the complexity parameter stops the tree from growing, though one can think of that as a sort of preemptive pruning. I'm looking for an explicit description of how that package uses the risk formula in deciding whether to split, but I can't yet find it. $\endgroup$
    – Sean Easter
    Commented Nov 2, 2015 at 22:44
  • $\begingroup$ One thing that can not find is how do you measure the quality of predictions in regression tree in the classification tree there is missclassification rate at each terminal node, but there is no option $extra=3$ in regression tree using $method="anova"$ in $rpart()$ $\endgroup$
    – user72621
    Commented Nov 2, 2015 at 22:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.