How to produce a CI for a value predicted in CART?

Question

My goal is to create CI for the CART prediction of new_x

Consider the following code:

require(rpart)    
set.seed(147830)
n <- 100
x1 <- runif(n)
x2 <- rnorm(n)
y <- x1 + 2*x2 + rnorm(n, 0, .5)
DAT <- data.frame(y,x1,x2)
fit <- rpart(y ~ x1 + x2, data = DAT)
new_x <- data.frame(x1 = .5 , x2 = .25)
predict(fit, newdata = new_x) # 1.353142 # "should" be ~1

The only way I can think of at the moment is to bootstrap on x1/x2 from DAT, build many prediction models with them, for each predict new_x, and then use the lower/upper percentiles.

Is there some other way to do it? What are the advantages/disadvantages of it?

Thanks.

Answer 1

Couldn't you use the distribution of the observed y in the leafs/terminal nodes to get an idea about the precision of the estimated mean for new observations falling into that leaf? For CIs with large coverage you'd probably need a lot of data / a small tree so that each terminal node contains a lot of observations, though.

i.e. do s.th like:

quantile(DAT$y[predict(fit) ==  predict(fit, newdata = new_x)], 
           probs=c(.025, .05, .1, .9, .95, .975))

maybe with a more elegant method than what I used to find out which of the original y fall into the same leaf as the observation you are forecasting.