5
$\begingroup$

I have a question of using the rpart for the regression tree. I am wondering when i use plotcp, where would my validation data comes from? Currently, I have a training data set, test set and validation set. But if I simply do plotcp, how do I speficy my validation set?

#Regression Tree Example
library(rpart)

# grow tree 
fit <- rpart(Mileage~Price + Country + Reliability + Type, 
   method="anova", data=cu.summary)

printcp(fit) # display the results 
plotcp(fit) # visualize cross-validation results 
summary(fit) # detailed summary of splits
$\endgroup$
6
$\begingroup$

The rpart package's plotcp function plots the Complexity Parameter Table for an rpart tree fit on the training dataset. You don't need to supply any additional validation datasets when using the plotcp function.

The Rpart implementation first fits a fully grown tree on the entire data $D$ with $T$ terminal nodes. After this step, the tree is pruned to the smallest tree with lowest miss-classification loss. This is how it works:

  1. The data is then split into $n$ (default = 10) randomly selected folds: $F_1$ to $F_{10}$
  2. It then uses 10-fold cross-validation and fits each sub-tree $T_1 ... T_m $ on each training fold $D_s$.
  3. The corresponding miss-classification loss (risk) $R_m$ for each sub-tree is then calculated by comparing the class predicted for the validation fold vs. actual class; and this risk value for each sub-tree is summed up for all folds.
  4. The complexity parameter $\beta$ giving the lowest total risk over the whole dataset is finally selected.
  5. The full data is then fit using this complexity parameter and this tree is selected as the best trimmed tree.

Hence, when you use plotcp, it plots the relative cross-validation error for each sub-tree from smallest to largest to let you compare the risk for each complexity parameter $\beta$.

For example, refer to the following fit using the StageC cancer prognosis data-set in the rpart package:

> printcp( cfit)

Classification tree:
rpart(formula = progstat ~ age + eet + g2 + grade + gleason + 
    ploidy, data = stagec, method = "class")

Variables actually used in tree construction:
[1] age    g2     grade  ploidy

Root node error: 54/146 = 0.36986

n= 146 

        CP nsplit rel error  xerror    xstd
1 0.104938      0   1.00000 1.00000 0.10802
2 0.055556      3   0.68519 1.16667 0.11083
3 0.027778      4   0.62963 0.96296 0.10715
4 0.018519      6   0.57407 0.96296 0.10715
5 0.010000      7   0.55556 1.00000 0.10802

The cross-validated error is scaled down for easier reading; the error bars on the plot show one standard deviation of the x-validated error.

enter image description here

References:

  1. An Introduction to Recursive Partitioning Using the RPART Routines, URL: https://cran.r-project.org/web/packages/rpart/vignettes/longintro.pdf
$\endgroup$
5
  • $\begingroup$ How can the missclassification error rate in the cross validation be bigger then 1? $\endgroup$ Jun 3 '19 at 21:23
  • 2
    $\begingroup$ It is a relative cross validation error. Please refer to the graph Y-axis label. $\endgroup$ Jun 4 '19 at 19:45
  • 1
    $\begingroup$ how i understand its the xerror from the printcp() function. When you say its the relative error what do you mean by relalitve? Relative to what? $\endgroup$ Jun 5 '19 at 14:44
  • $\begingroup$ It plots the relative cross-validation error for each sub-tree from smallest to largest to let you compare the risk for each complexity parameter. $\endgroup$ Jun 5 '19 at 14:53
  • $\begingroup$ do you mean the third oder the fourth collum of the of pringtcp() output and what is the relative cross validation error, how it is computed? i thought the cross validation error is the missclassification error rate but this seems to be wrong... $\endgroup$ Jun 5 '19 at 14:57

Not the answer you're looking for? Browse other questions tagged or ask your own question.