0
$\begingroup$

My experience with R, rpart model is the model seems to be always too simple / over pruning. In many experiments, I am seeing a over simplified tree with only 1 or 2 split, but I know from the domain knowledge the model should be more complicated.

Currently, I am manually experimenting with cp and minsplit parameter to get a more complicated tree, sometimes sacrifice cross validation performance to get a more "informative" tree.

Would other models like conditional inference trees be better in terms of over pruning? Or other suggestions to get a more complicated model?

PS, I aware of over-fitting risk, but think the model I got from CART is still too simple. And I remember I read from somewhere that the pruning method proposed in CART may have the problem of "over pruning".

Improve this question
$\endgroup$
  • $\begingroup$ Is there any particular reason you're not using a random forest? $\endgroup$ – Sycorax♦ Nov 28 '16 at 16:19
  • $\begingroup$ @Sycorax Need to extract some knowledge from tree, to present to business. Executives do not like "black box" model ... $\endgroup$ – Haitao Du Nov 28 '16 at 16:20
  • $\begingroup$ I am not familiar with rpart but the original CART program of Brieman,, Friedman et al. which is now available from Salford systems may have a parameter that controls the degree of splitting. I do not know if the program has a default value for the parameter. The complexity of the tree that you construct depends on your sample size. So if the sample size is small it may be difficult to generate splits that lead to terminal nodes with more than 1 or 2 cases. $\endgroup$ – Michael R. Chernick Nov 28 '16 at 16:54
  • $\begingroup$ @hxd1011: Your comment suggests that you're interested in feature importance of trees. See this answer: stats.stackexchange.com/questions/6478/… $\endgroup$ – Alex R. Nov 28 '16 at 18:30
  • $\begingroup$ @AlexR. thanks for the link. In addition to importance, I still want to have a "reasonable complicated tree" to show, that is why I ask this. $\endgroup$ – Haitao Du Nov 28 '16 at 18:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.