# Confidence interval for Regression Trees

Many people think the regression tree is only an algorithm and it doesn't make sense approach confidence interval to it so I'd like to know if there's anyone figured out how to do it.

A regression tree can be write as follows (Friedman (2000)):

$$f(x)=\sum_{j=1}^{K}b_j I(x \in R_j)$$

Where $$R_j$$ is subset of predictors train set belongs to node $$j$$ that is $$x \in R_j$$ if $$x$$ respect all rules that generated $$j$$-th node, $$b_j$$ is the mean of response to $$x \in R_j$$, $$I(x \in R_j)=1$$ if $$x \in R_j$$ and $$I(x \in R_j)=0$$ unlike.

A var of regression tree can be get as follows:

$$V(f(x))=\sum_{j=1}^{K}V(b_j)I(x \in R_j)$$ since $$y_i$$ is iid and $$I(x \in R_j)^2=I(x \in R_j)$$

$$V(f(x))=\sum_{j=1}^{K}V(\frac{\sum_{i=1}^{|R_j|}y_i}{|R_j|})I(x \in R_j)$$ where $$|R_j|$$ is $$R_j's$$ size.

$$V(f(x))=\sum_{j=1}^{K}\frac{\sum_{i=1}^{|R_j|}V(y_i)}{|R_j|^2}I(x \in R_j)$$

$$V(f(x))=\sum_{j=1}^{K} \frac{\sum_{i=1}^{|R_j|}\sigma^2}{|R_j|^2}I(x \in R_j)$$

$$V(f(x))=\sum_{j=1}^{K}\frac{\sigma^2}{|R_j|}I(x \in R_j)$$

If $$y_i\sim N(u_y,\sigma^2)$$ then $$f(x) \sim N(\sum_{j=1}^{K}u_yI(x \in R_j), \sum_{j=1}^{K}\frac{\sigma^2}{|R_j|}I(x \in R_j))$$ so we can get a confidence interval to $$f(x)$$ by using normal distribuition theory.

• $|R_j|$ must be greater than 30 to asymptotically approach works – Davi Américo Jun 16 at 20:54