4
$\begingroup$

I am new to R and using rpart for building a regression tree for my data.I wanted to use all the input variables for building the tree, but the rpart method using only a couple of inputs as shown below. As we can see, I have provided 10 inputs, but rpart used only two inputs. Please let me know how can force rpart method to use all the input variables. Thanks.

rm = rpart(uloss ~ tc_b + ublkb + mpa_a + mpa_b + 
     sys_a + sys_b + usr_a, data = data81, method="anova")
> princtp(rm)  

Regression tree:
rpart(formula = uloss ~ tc_b + ublkb + mpa_a + mpa_b + sys_a + 
    sys_b, data = data81, weights = usr_a, method = "anova")

Variables actually used in tree construction:
[1] mpa_a tc_b     
Root node error: 647924/81 = 7999
n= 81    
       CP nsplit rel error  xerror     xstd
1 0.403169      0   1.00000 1.04470 0.025262
2 0.092390      1   0.59683 0.66102 0.015238
3 0.081084      2   0.50444 0.70702 0.013123
4 0.045304      3   0.42336 0.58683 0.012129
5 0.010000      4   0.37805 0.51930 0.011942

One more question:

I have used rpart.control for minsplit=2, and got the following for another data. Inorder to avoid overfititng the data, do I need to use splits 3 or splits 7. Shouldn't I use splits 7? Please let me know.

Variables actually used in tree construction: [1] ct_a ct_b usr_a

Root node error: 23205/60 = 386.75

n= 60

        CP nsplit rel error  xerror     xstd
1 0.615208      0  1.000000 1.05013 0.189409
2 0.181446      1  0.384792 0.54650 0.084423
3 0.044878      2  0.203346 0.31439 0.063681
4 0.027653      3  0.158468 0.27281 0.060605
5 0.025035      4  0.130815 0.30120 0.058992
6 0.022685      5  0.105780 0.29649 0.059138
7 0.013603      6  0.083095 0.21761 0.045295
8 0.010607      7  0.069492 0.21076 0.042196
9 0.010000      8  0.058885 0.21076 0.042196
$\endgroup$
6
$\begingroup$

Perhaps you misunderstood the message? It is saying that, having built the tree using the control parameters specified, only the variables mpa_a and tc_b have been involved in splits. All the variables were considered, but just these two were needed.

That tree seems quite small; do you have only a small sample of observations? If you want to grow a bigger tree for subsequent pruning back, then you need to alter the minsplit and minbucket control parameters. See ?rpart.control, e.g.:

rm <- rpart(uloss ~ tc_b + ublkb + mpa_a + mpa_b + 
            sys_a + sys_b + usr_a, data = data81, method = "anova",
            control = rpart.control(minsplit = 2, minbucket = 1))

would try to fit a full tree --- but it will be hopelessly over-fitted to the data and you must prune it back using prune(). However, that might assure you that rpart() used all the data.

$\endgroup$
3
  • 1
    $\begingroup$ @kkp the convention is to use the smallest tree within one standard error of the best tree. The best tree is in row 8 (7 splits), but the tree in row 7 (6 splits) does effectively the same job and is simpler, hence the 1 standard error rule would select it. Note Frank's comment in his Answer. Almost certainly you want to be using one of the bagged/boosted trees or random forest algorithms. $\endgroup$ – Gavin Simpson Jul 25 '11 at 16:34
  • $\begingroup$ according to the above rpart results, rpart() method used finally two inputs to predict the response variable. That is, according to rpart(), these two inputs are only important and used for classification. Am I right? $\endgroup$ – samarasa Jul 26 '11 at 22:57
  • $\begingroup$ Yes you are right samarasa $\endgroup$ – alily Aug 17 '16 at 8:30
5
$\begingroup$

If the number of observations is less than around 20,000 the trees built by rpart do not have a reliable structure. That is, if you were to use the bootstrap to repeat the process, you will see many different trees that are called 'optimal'.

$\endgroup$
3
  • $\begingroup$ Since you qualified your 20,000 observations with "trees built by rpart`, are there any of these single-tree algorithms that don't need upwards of 20,000 observations? Or did you mean any single-tree based methodology (and hence the need to use boosted or bagged trees or random forests?)? $\endgroup$ – Gavin Simpson Jul 25 '11 at 15:46
  • $\begingroup$ I don't know of a single tree algorithm that works with non-huge datasets. That's why random forests, bagging, and boosting are so popular. But you lose the interpretability of the tree with these. In general there is a interpretability-reliability tradeoff. You might as well fit a rich regression model with appropriate attention to number of parameters examined. Regression models are interpretable and you can make them reliable. A good start is usually a flexible additive regression model, allowing many of the continuous predictors to have nonlinear effects (using e.g. splines). $\endgroup$ – Frank Harrell Jul 25 '11 at 16:19
  • $\begingroup$ thanks, was just checking - that was my interpretation of single-tree based methods. $\endgroup$ – Gavin Simpson Jul 25 '11 at 17:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.