CART - Classification And Regression Trees

I try to prune one regression tree build with the rpart function in R. To decide where to prune the tree I used the plotcp function. But I noticed that if I use the same predictor variables and in the same order the plotcp graph always change? How is this possible? Thank you for your explanation in advance.

• @whuber, I recommend to make "cart" tag a junior synonym of "decision-tree" or "classification-tree". Because, often CART acronym is regarded to be just a particular method or algorithm of the tree, along with other methods such as CHAID or QUEST. – ttnphns Apr 13 '13 at 8:07

If you re-run just plotcp you should get the same exact plot. But if you re-run rpart you will get different fits because randomization is involved. You can avoid this by setting a seed before each run of rpart.

e.g.

fit1 <- rpart(Kyphosis ~ Age + Number + Start, data=kyphosis)
plotcp(fit1)

fit2 <- rpart(Kyphosis ~ Age + Number + Start, data=kyphosis)
plotcp(fit2)


will yield two different trees and thus two different fits, and the plot will reflect that. But

plotcp(fit2)
plotcp(fit2)
plotcp(fit2)


should be identical, as should,

set.seed(10020101)
fit1 <- rpart(Kyphosis ~ Age + Number + Start, data=kyphosis)
plotcp(fit1)

set.seed(10020101)
fit2 <- rpart(Kyphosis ~ Age + Number + Start, data=kyphosis)
plotcp(fit2)

• Where is the random part in RPART algorithm? I thought that the splits are computed in exact way. What is random? – pikachu Dec 1 '19 at 12:41
• I am not sure where the random part comes in, but it must be there, as keeping the seed constant makes the output constant. However, the rpart part stays the same; the random element is somewhere in plotcp. – Peter Flom - Reinstate Monica Dec 1 '19 at 13:24
• I have found out: for computation of the xerror and xstd it uses cross validation and splitting data set to folds for CV is random. So decision tree is indeed computed in exact way, but estimation of error and its std is random. – pikachu Dec 1 '19 at 13:44

This analysis may be done when regression trees include controls which allow you to limit which variables are allowed to enter the tree at any specified depth of the tree. Suppose we had the example of students in schools in districts with variables relevant to each level in the data. Then we might organize a tree so that it follows a pattern like:

Depth 0 through 2: splitters are district characteristics Depth 3 and 4: splitters are school characteristics Depth 5 and below: splitters are student characteristics

Of course we cannot know in advance where to draw the boundaries in the tree so experimentation would be mandatory....and I would invite more people from the forum to jump in and share with us views about statistical rules of thumb of how to solve this...

With regular CART you could start three analyses each using the relevant predictors for that level.

So start with district predictors and build tree. Save predicted scores and residuals.

Take residuals as new dependent variable and use only school variables. Save predicted vales and residuals.

Take second stage residuals as dependent variable and use student vars as predictors. Save predictions and residuals.

Final predictions are sum of the predictions of the three trees.