5
$\begingroup$

I am running a regression tree using rpart and I would like to understand how well it is performing.

I know that rpart has cross validation built in, so I should not divide the dataset before of the training.

Now, I build my tree and finally I ask to see the cp.

> fit <- rpart(slope ~ ., data = ph1)
> printcp(fit)

Regression tree:
rpart(formula = slope ~ ., data = ph1)

Variables actually used in tree construction:
[1] blocksize dimension maps      reducers 

Root node error: 8.9483/364 = 0.024583

n= 364 

        CP nsplit rel error  xerror     xstd
1 0.517156      0   1.00000 1.00305 0.095998
2 0.155374      1   0.48284 0.48686 0.047503
3 0.116019      2   0.32747 0.37237 0.034623
4 0.029928      3   0.21145 0.22534 0.021952
5 0.018020      4   0.18152 0.21134 0.021075
6 0.016643      5   0.16350 0.20052 0.021303
7 0.015986      7   0.13022 0.18776 0.021119
8 0.010000      8   0.11423 0.15334 0.016906

Now I don't follow anymore.

What are those number?

If it was a classification I could follow those number thanks to this question

But what about a regression tree ?

The test sample is here

Improve this question
$\endgroup$
2
  • $\begingroup$ What data are ph1? Please add a reproducible example for people to work with. $\endgroup$
    – gung - Reinstate Monica
    May 29, 2016 at 16:48
  • $\begingroup$ @lmo, no, there are just 8... $\endgroup$
    – Siscia
    May 29, 2016 at 21:11

3 Answers 3

Reset to default
5
$\begingroup$

CP table is the most important part of the RPART, it gives the complexity of the tree model (cp column) training error (rel error) and cross validation error (xerror).

I have a set of notes on how every numbers are calculated. But I am running a regression on the mtcar data set. Note directly to your question but I think it can answer your question well. Sorry the annotation might be little messy.

enter image description here

enter image description here

I would suggest you to read RPART manual Page 20. And if possible the original cart book.

Improve this answer
$\endgroup$
1
  • $\begingroup$ your answer is excellent +1. I like your links particularly the ,classic book by Breiman, Friedman, Olshen and Stone.. $\endgroup$
    – Michael R. Chernick
    Apr 8, 2017 at 4:23
0
$\begingroup$

You can use the sperrorest package to estimate the performance of your model. Various error measures are returned.

The following code is taken from ?sperrorest. You are welcome to modify pred.fun to your needs. This is just an example.

The actual code syntax refers to version 1.0.0 which is currently available from Github using devtools::install_github("pat-s/sperrorest"). 

data(ecuador) # Muenchow et al. (2012), see ?ecuador
fo <- slides ~ dem + slope + hcurv + vcurv + log.carea + cslope

# Example of a classification tree fitted to this data:
library(rpart)
ctrl <- rpart.control(cp = 0.005) # show the effects of overfitting
fit <- rpart(fo, data = ecuador, control = ctrl)
par(xpd = TRUE)
plot(fit, compress = TRUE, main = 'Stoyans landslide data set')
text(fit, use.n = TRUE)

# Non-spatial 5-repeated 10-fold cross-validation:
mypred.rpart <- function(object, newdata) predict(object, newdata)[, 2]
nspres <- sperrorest(data = ecuador, formula = fo,
                     model.fun = rpart, model.args = list(control = ctrl),
                     pred.fun = mypred.rpart,
                     smp.fun = partition.cv, 
                     smp.args = list(repetition = 1:5, nfold = 10))
summary(nspres$error.rep)                    
summary(nspres$error.fold)

                      mean          sd       median         IQR
train.auroc       7.465106e-01 0.005186850 7.478738e-01 0.003922117
train.error       2.953111e-01 0.005235886 2.952222e-01 0.004166667
train.accuracy    7.046889e-01 0.005235886 7.047778e-01 0.004166667
train.sensitivity 6.742444e-01 0.011529458 6.727778e-01 0.016000000
train.specificity 7.351333e-01 0.005994030 7.361111e-01 0.007111111
train.fpr70       3.115111e-01 0.017166685 3.202222e-01 0.023777778
train.fpr80       4.801778e-01 0.022528528 4.900000e-01 0.035222222
train.fpr90       6.819556e-01 0.015305853 6.808889e-01 0.019555556
train.tpr80       5.599111e-01 0.016436447 5.530000e-01 0.023000000
train.tpr90       3.345556e-01 0.013286259 3.297778e-01 0.022444444
train.tpr95       1.702667e-01 0.017200632 1.728889e-01 0.004666667
train.events      9.000000e+03 0.000000000 9.000000e+03 0.000000000
train.count       1.800000e+04 0.000000000 1.800000e+04 0.000000000
test.auroc        6.446438e-01 0.008173371 6.446755e-01 0.003872000
test.error        3.863000e-01 0.004590752 3.850000e-01 0.006500000
test.accuracy     6.137000e-01 0.004590752 6.150000e-01 0.006500000
test.sensitivity  5.890000e-01 0.018179659 5.960000e-01 0.021000000
test.specificity  6.384000e-01 0.012601587 6.350000e-01 0.020000000
test.fpr70        5.156000e-01 0.020634922 5.190000e-01 0.034000000
test.fpr80        6.286000e-01 0.018447222 6.360000e-01 0.017000000
test.fpr90        7.910000e-01 0.018547237 7.880000e-01 0.008000000
test.tpr80        3.978000e-01 0.009364828 3.970000e-01 0.004000000
test.tpr90        1.844000e-01 0.012239281 1.870000e-01 0.014000000
test.tpr95        8.560000e-02 0.020767763 8.800000e-02 0.025000000
test.events       1.000000e+03 0.000000000 1.000000e+03 0.000000000
test.count        2.000000e+03 0.000000000 2.000000e+03 0.000000000
Improve this answer
$\endgroup$
-1
$\begingroup$

All the details for the CP calculation with numerical examples are show here https://sites.google.com/stern.nyu.edu/rdeo/home

Improve this answer
$\endgroup$
2
  • 2
    $\begingroup$ Welcome to the site. At present this is more of a comment than an answer. You could expand it, perhaps by giving a summary of the information at the link, or we can convert it into a comment for you. $\endgroup$
    – gung - Reinstate Monica
    Jan 15, 2020 at 18:50
  • 1
    $\begingroup$ Welcome to CV. Because duplicate answers do not fit within this site's aims, I have deleted your other two identical posts. Please see our help center for more information and guidance. $\endgroup$
    – whuber ♦
    Jan 15, 2020 at 19:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.