Prediction interval for single future response

Question

I know the following results:

$\frac{RSS}{\sigma^{2}} \sim \chi^{2}_{n-r}$ where $r = rank(X)$ and $k$ is the degrees of freedom. Then, for any $c \in \mathbb{R^{p}} $, and when X has full rank, i.e. $rank(X) = p$, $\frac{\mathbf{c}^{T}\hat{\beta}-\mathbf{c}^{T}\beta}{\sqrt{\mathbf{c^{T}(X^{T}X)^{-1}\mathbf{c}\frac{RSS}{n-p}}}} \sim t_{n-p}$, following from $\frac{\mathbf{c^{T}}\hat{\beta}-\mathbf{c^{T}}\beta}{\mathbf{c^{T}}(X^{T}X)^{-1}\mathbf{c}\sigma^{2}} \sim N(0,1)$.

Suppose that we fit a linear model with design matrix X to obtain estimates $\boldsymbol{\hat{\beta}}$ and $\hat{\sigma}^{2}$. For given covariates, $x_{*}$, the predicted response is $\hat{y}_{*} = x_{*}^{T}\boldsymbol{\hat{\beta}}$.

We also know that $Var(x_{*}^{T}\hat{\beta}) = x_{*}^{T}(X^{T}X)^{-1}x_{*}\sigma^{2}$.

How, from the above, can I get the following result? A $100(1-\alpha)$ confidence interval for a single future response is $\hat{y_{*}} \pm t_{n-p}^{(\frac{\alpha}{2})}\hat{\sigma}\sqrt{1+x_{*}^{T}(X^{T}X)^{-1}x_{*}}$? I've been stuck for ages.

Answer 1

If you wonder where did the extra $1$ come from. We have

$$E(\hat{Y}_* - Y_*)^2 = Var(\hat{Y}_*) + E(E(\hat{Y}_*) - Y_*)^2 = Var(\hat{Y}_*) + E(Y_* - x_*\beta)^2 = Var(\hat{Y}_*) + \sigma^2$$

Let's denote $Z = Y_* - \hat{Y}_*$. From Fisher's Theorem we know that $\hat{Y}_*$ is normally distributed, then so is $Z$. Now

$$\frac{Z - E(Z)}{\sqrt{Var(Z)}} = \frac{Y_* - \hat{Y}_* - 0}{\sqrt{E(\hat{Y}_* - Y_* - 0)^2}} = \frac{Y_* - \hat{Y}_*}{\sqrt{\sigma^2D_*}} \sim N(0,1)$$

Where $D_* = 1 + x_*^T(X^TX)^{-1}x_*$. Substituing estimator of $\sigma^2$, $S^2 = \frac{RSS}{n-p}$

$$\frac{Y_* - \hat{Y}_*}{\sqrt{D_*\frac{RSS}{n-p}}} \sim t(n-p)$$

and finally

$$P\Big(t_{\alpha/2} \le \frac{Y_* - \hat{Y}_*}{\sqrt{D_*\frac{RSS}{n-p}}} \le t_{1-\alpha/2}\Big) = 1 - \alpha$$

$$P\Big(\hat{Y}_*-t_{1-\alpha/2}\sqrt{D_*\frac{RSS}{n-p}} \le Y_* \le \hat{Y}_*+t_{1-\alpha/2}\sqrt{D_*\frac{RSS}{n-p}}\Big) = 1 - \alpha$$

Answer 2

Prediction intervals are wider because there is that extra uncertainty due to the new random observation. See Hahn and Meeker's Statistical Intervals text published by Wiley for clear derivations and discussions of the various types of statistical intervals including Bayesian credible regions.