Is there an ANOVA table generalization for two nested CART models?

Question

The last line is an example of what I'm looking for:

data(airquality)
attach(airquality)

lm1 <- lm(Ozone ~ Solar.R+Wind)
lm2 <- lm(Ozone ~ Solar.R+Wind+Temp)
anova(lm1 , lm2)

require(rpart)
rp1 <- rpart(Ozone ~ Solar.R+Wind)
rp2 <- rpart(Ozone ~ Solar.R+Wind+Temp)
anova(rp1 , rp2) # this doesn't exist - is there something like it? some sort of anova.rpart function?

Thanks!

Answer 1

Just to add to Frank's points and paint a somewhat finer picture: CART/RPART is indeed highly exploratory and adding a p-value is difficult. I have seen some rare cases where people tried to use bootstrapping to get such p-value but I agree with Frank that it's not worth the effort.

As for combining statistical inference with recursive partitioning more generally: The CTree and MOB methods implemented in the "party" package as well as several other tools outside R (especially the work of Loh & co-authors) tries to combine standard statistical tests (nonparametric in case of CTree, parametric for MOB) for growing the trees. They also control their error level in the sense of a closed testing procedure. However, the inference for the final fitted tree is still not trivial. The situation is similar to a regression model where you have selected interactions between your regressors in a forward search (and we know how Frank feels about that). Hence, also the "party" package has no anova() methods. We do provide AIC() for "mob" objects, though. It is not strictly valid (because we haven't done full optimization of the log-likelihood but only forward search) but it would be conservative. So the error is into the preferable direction.

Answer 2

Recursive partitioning does not provide such inferential statistics. It is a highly exploratory method that would require an enormous multiplicity adjustment should you compute regression and error sum of squares from the result. Better would be to do formal but flexible modeling of the two predictors, e.g., using regression splines, and if allowing for interaction, tensor splines (by adding products of spline terms). There is a good reason there is no anova.rpart function in the R rpart package.