4
$\begingroup$

I'm building a Decision Tree on the iris dataset using R Studio. I take 100 observations as training data, and 50 as testing one. I train the tree model and plot it using these commands:

treeModel <- ctree(Species ~ ., data = trainData)
plot(treeModel)

And here is the result I got: Decision Tree

I have 2 questions:

  1. What is the purpose of the p-value inside each node? What is the test?
  2. How does it choose the split point? For example, why did it take 1.9 as a split point of Petal.Length?

Thank you so much for your help.

Improve this question
$\endgroup$
0

1 Answer 1

Reset to default
6
$\begingroup$

This is mostly explained in the documentation for ctree. Type ?ctree. The most relevant part is:

Roughly, the algorithm works as follows: 1) Test the global null hypothesis of independence between any of the input variables and the response (which may be multivariate as well). Stop if this hypothesis cannot be rejected. Otherwise select the input variable with strongest association to the response. This association is measured by a p-value corresponding to a test for the partial null hypothesis of a single input variable and the response. 2) Implement a binary split in the selected input variable. 3) Recursively repeat steps 1) and 2).

So the independence of each variable with the class (Species) is tested. The split is made on the variable with the lowest p-value. They are reporting the p-value so you can see how strong the association is (why that variable was chosen). Unfortunately, they do not seem to document the method used to choose the split point. However, some methods choose the splitting point that optimizes some measure of node purity after the split. For example, this paper

Usama M. Fayyad, Keki B. Irani: Multi-interval discretization of continuous valued attributes for classification learning. In: Thirteenth International Joint Conference on Artificial Intelligence, 1022-1027, 1993.

finds the cut point that maximizes the change in entropy.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Based on the document, the null hypothesis is an input variable does not associate with the response. Is that correct? $\endgroup$
    – Triet Doan
    Commented Jan 8, 2017 at 18:00
  • $\begingroup$ @AnhTriet Yes. the variable with the strongest evidence for rejecting that hypothesis is the one that is selected. $\endgroup$
    – G5W
    Commented Jan 8, 2017 at 18:04
  • $\begingroup$ I thought it would choose the node base on the Information Gain? $\endgroup$
    – Triet Doan
    Commented Jan 8, 2017 at 18:07
  • $\begingroup$ C4.5 (and its implementation J48) use Information Gain, but not all decision tree models do. CART uses GINI. At least according to the documentation, ctree uses this way to decide. If you want to use Information Gain, there is a package called C50 for R that builds trees using IG to decide which variable to split on. $\endgroup$
    – G5W
    Commented Jan 8, 2017 at 18:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.