5
$\begingroup$

When building a CART model (specifically classification tree) using rpart (in R), it is sometimes obvious that there are variables (X's) that are meaningful for predicting some of the outcome (y) variables - while other predictors are relevant for other y's only.

How can it be estimated, which explanatory variable is "used" for which of the predicted value in the outcome variable?

Here is an example code in which x2 is the only important variable for predicting "b" (one of the y outcomes). There is no predicting variable for "c", and x1 is a predictor for "a", assuming that x2 permits it.

How can this situation be extracted from the fitted model?

N <- 200
set.seed(5123)
x1 <- runif(N)
x2 <- runif(N)
x3 <- runif(N)
y <- sample(letters[1:3], N, T)
y[x1 <.5] <- "a"
y[x2 <.1] <- "b"

fit <- rpart(y ~ x1+x2)
fit2 <- prune(fit, cp= 0.07)
plot(fit2)
text(fit2, use.n=TRUE)

Thanks.

$\endgroup$
  • 1
    $\begingroup$ If x1 < .5 AND x2 < .1, then y="b". Is that the intended behaviour? (i.e., is the order of lines 7 and 8 significant?). $\endgroup$ – F. Tusell Dec 14 '10 at 8:27
  • $\begingroup$ Hi F Tusell - it is significant since this order creates a behavior were a is dependent on x1 and x2, while b only depends on x2. $\endgroup$ – Tal Galili Dec 14 '10 at 12:21
1
$\begingroup$

Tal,

My understanding based on this article, is that you cannot obtain the individual variables related to each separate class when the class is a factor. However, using rpart.control, you are able to identify the variables that are important at each node.

http://cran.r-project.org/web/packages/caret/vignettes/caretVarImp.pdf

Recursive Partitioning: The reduction in the loss function (e.g. mean squared error) attributed to each variable at each split is tabulated and the sum is returned. Also, since there may be candidate variables that are important but are not used in a split, the top competing variables are also tabulated at each split. This can be turned on using the maxcompete argument in rpart.control. This method does not currently provide class specific measures of importance when the response is a factor.

$\endgroup$
  • $\begingroup$ Thanks "user". This doesn't answer my question - but it is a valuable reference for me - thank you. $\endgroup$ – Tal Galili Dec 19 '10 at 8:57
0
$\begingroup$

I think your answer is in CART model. In the example you provided, some of the values in y is completely random, and you assigned some values according to some criteria in x1 and x2. I don't think you can find a SINGLE variable to predict one certain value in y.

I replicated your data and make it a data frame, then filter it by $x_1\geq.5$. You can see the data doesn't follow your assumptions, that is, $x_1 <.5$ $\Rightarrow$ $a$ $\&$ $x_2<.1$ $\Rightarrow$ $b$.

enter image description here

From your data, there should not be any one predictor that can be surely linked to certain outcome. CART tells you the classification criteria for each values in y. The variance importance mentioned by "user2238" tells you, out of the model ($y\tilde{} x_1+x_2$), which is more important. Hope this helps. And comments are also welcome.

$\endgroup$
  • $\begingroup$ Welcome to cross validate! can you please add the R output as a code snippet instead a picture? Just paste it in, select it, and click the {} symbol in the top pane. Its easier to read, search and copy. $\endgroup$ – Yuval Spiegler Dec 14 '16 at 12:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.