5
$\begingroup$

I am new to R and using rpart for building a regression tree for my data.I wanted to use all the input variables for building the tree, but the rpart method using only a couple of inputs as shown below. As we can see, I have provided 10 inputs, but rpart used only two inputs. Please let me know how can force rpart method to use all the input variables. Thanks.

rm = rpart(uloss ~ tc_b + ublkb + mpa_a + mpa_b + 
     sys_a + sys_b + usr_a, data = data81, method="anova")
> princtp(rm)  

Regression tree:
rpart(formula = uloss ~ tc_b + ublkb + mpa_a + mpa_b + sys_a + 
    sys_b, data = data81, weights = usr_a, method = "anova")

Variables actually used in tree construction:
[1] mpa_a tc_b     
Root node error: 647924/81 = 7999
n= 81    
       CP nsplit rel error  xerror     xstd
1 0.403169      0   1.00000 1.04470 0.025262
2 0.092390      1   0.59683 0.66102 0.015238
3 0.081084      2   0.50444 0.70702 0.013123
4 0.045304      3   0.42336 0.58683 0.012129
5 0.010000      4   0.37805 0.51930 0.011942

One more question:

I have used rpart.control for minsplit=2, and got the following for another data. Inorder to avoid overfititng the data, do I need to use splits 3 or splits 7. Shouldn't I use splits 7? Please let me know.

Variables actually used in tree construction: [1] ct_a ct_b usr_a

Root node error: 23205/60 = 386.75

n= 60 

        CP nsplit rel error  xerror     xstd
1 0.615208      0  1.000000 1.05013 0.189409
2 0.181446      1  0.384792 0.54650 0.084423
3 0.044878      2  0.203346 0.31439 0.063681
4 0.027653      3  0.158468 0.27281 0.060605
5 0.025035      4  0.130815 0.30120 0.058992
6 0.022685      5  0.105780 0.29649 0.059138
7 0.013603      6  0.083095 0.21761 0.045295
8 0.010607      7  0.069492 0.21076 0.042196
9 0.010000      8  0.058885 0.21076 0.042196
Improve this question
$\endgroup$

2 Answers 2

Reset to default
7
$\begingroup$

Perhaps you misunderstood the message? It is saying that, having built the tree using the control parameters specified, only the variables mpa_a and tc_b have been involved in splits. All the variables were considered, but just these two were needed.

That tree seems quite small; do you have only a small sample of observations? If you want to grow a bigger tree for subsequent pruning back, then you need to alter the minsplit and minbucket control parameters. See ?rpart.control, e.g.:

rm <- rpart(uloss ~ tc_b + ublkb + mpa_a + mpa_b + 
            sys_a + sys_b + usr_a, data = data81, method = "anova",
            control = rpart.control(minsplit = 2, minbucket = 1))

would try to fit a full tree --- but it will be hopelessly over-fitted to the data and you must prune it back using prune(). However, that might assure you that rpart() used all the data.

Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ @kkp the convention is to use the smallest tree within one standard error of the best tree. The best tree is in row 8 (7 splits), but the tree in row 7 (6 splits) does effectively the same job and is simpler, hence the 1 standard error rule would select it. Note Frank's comment in his Answer. Almost certainly you want to be using one of the bagged/boosted trees or random forest algorithms. $\endgroup$
    – Gavin Simpson
    Jul 25, 2011 at 16:34
  • $\begingroup$ according to the above rpart results, rpart() method used finally two inputs to predict the response variable. That is, according to rpart(), these two inputs are only important and used for classification. Am I right? $\endgroup$
    – samarasa
    Jul 26, 2011 at 22:57
  • $\begingroup$ Yes you are right samarasa $\endgroup$
    – alily
    Aug 17, 2016 at 8:30
5
$\begingroup$

If the number of observations is less than around 20,000 the trees built by rpart do not have a reliable structure. That is, if you were to use the bootstrap to repeat the process, you will see many different trees that are called 'optimal'.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Since you qualified your 20,000 observations with "trees built by rpart`, are there any of these single-tree algorithms that don't need upwards of 20,000 observations? Or did you mean any single-tree based methodology (and hence the need to use boosted or bagged trees or random forests?)? $\endgroup$
    – Gavin Simpson
    Jul 25, 2011 at 15:46
  • $\begingroup$ I don't know of a single tree algorithm that works with non-huge datasets. That's why random forests, bagging, and boosting are so popular. But you lose the interpretability of the tree with these. In general there is a interpretability-reliability tradeoff. You might as well fit a rich regression model with appropriate attention to number of parameters examined. Regression models are interpretable and you can make them reliable. A good start is usually a flexible additive regression model, allowing many of the continuous predictors to have nonlinear effects (using e.g. splines). $\endgroup$
    – Frank Harrell
    Jul 25, 2011 at 16:19
  • $\begingroup$ thanks, was just checking - that was my interpretation of single-tree based methods. $\endgroup$
    – Gavin Simpson
    Jul 25, 2011 at 17:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.