Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 5 down vote favorite

In the rpart() routine to create CART models, you specify the complexity parameter to which you want to prune your tree. I have seen two different recommendations for choosing the complexity parameter:

  1. Choose the complexity parameter associated with the minimum possible cross-validated error. This method is recommended by Quick-R and HSAUR.

  2. Choose the greatest complexity parameter whose estimated cross-validated error is still within a SE of the minimum possible cross-validated error. This is my interpretation of the package documentation, which says: "A good choice of cp for pruning is often the leftmost value for which the mean lies below the horizontal line" in reference to this plot.

The two choices of cp produce quite different trees in my dataset.

It seems that the first method will always produce a more complex, potentially overfitted, tree. Are there other advantages, disadvantages, recommendations in the literature, etc. I should take into account when deciding which method to use? I can provide more information about my particular modelling problem if that would be useful, but am trying to keep this question broad enough to be relevant to others.

share|improve this question
What does the horizontal line in the plot represent? – Bogdanovist Oct 9 '12 at 2:09
I believe it represents 1 SE above the minimum possible cross-validated error. – half-pass Oct 9 '12 at 3:12
If you have enough data, you can try separating it into a training and test data set, even for trees. This may be especially useful if you are mainly interested in prediction, as the test data set will give a good estimate of that. Another choice is the party package which uses significance tests (not usually something I recommend, but it seems relevant here). As always, though, the best test is usefulness and sense; this is especially true if you are mainly interested in explanation. – Peter Flom Oct 9 '12 at 10:49
My apologies for the slow response. To clarify, I thought that, by using cross-validation to compute the error at different tree sizes, I had already effectively split the data repeatedly into training and test sets. Would performing another training/test split be redundant in that case? Am I misunderstanding you? – half-pass Oct 20 '12 at 15:10

Know someone who can answer? Share a link to this question via email, Google+, Twitter, or Facebook.

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.