How do I derive prediction intervals for a general linear model?

My general linear model written in matrix form is,

$$ \mathbf{Y} = \mathbf{X} \mathbf{B} + \mathbf{R}$$

with each of the rows of the residual matrix $\mathbf{R}$, distributed as $\mathbf{r}_i \sim \mathcal{N}(\mathbf{0},\Sigma_{R})$. Alternatively, $row(\mathbf{R}) \sim \mathcal{N}(\mathbf{0}, \mathbf{I} \otimes \Sigma_{R})$.

I have a least squares estimator as

$$\hat{\mathbf{B}} = (\mathbf{X}^{T} \mathbf{X})^{-1} \mathbf{X}^{T} \,, $$

from affine transformations on $\mathbf{R}$, this estimator is distributed as,

$$ row(\hat{\mathbf{B}}) \sim \mathcal{N}(\mathbf{B}, \Sigma_{\hat{\mathbf{B}}}) $$

What I wish to do is form a prediction interval on a new instance of $\mathbf{Y}$ that is within the range (an interpolation, not extrapolation). This new instance is $\tilde{\mathbf{y}}_{i}$. What I wish to do is find a prediction interval (or confidence interval?) on this new instance.

On my own, I have used affine transformations again on $\hat{\mathbf{B}}$ to find that

$$\tilde{\mathbf{y}}_{i} \sim \mathcal{N}(\tilde{\mathbf{x}} \hat{\mathbf{B}}, \mathbf{X}^{T} (\mathbf{X}^{T} \mathbf{X})^{-1} \mathbf{X} \otimes \hat{\Sigma}_{R} ) $$

However I am uncertain in my solution. My textbooks on regular multiple linear regression have formulas for prediction intervals. But if the LS estimator has a distribution, why not just use this result? From it I can make $1\sigma$ or $3 \sigma$ intervals etc. In short,

$\textbf{How does one derive prediction intervals for the general linear model? }$

  • $\begingroup$ The formula for iid errors is given at stats.stackexchange.com/questions/9131. Although your question refers to the "general" linear model, the estimator you offer is not the one given by the GLM--it is the OLS estimator which is appropriate only for constant diagonal $\Sigma_R$. This calls into question what you are really looking for. Could you make this a little clearer? $\endgroup$ – whuber Feb 19 '14 at 21:27
  • $\begingroup$ $row(\mathbf{R})$ is block diagonal, while $\Sigma_{R}$ is not. OLS is appropriate since each vector $\mathbf{r}_{i}$ is iid distributed... right? $\endgroup$ – bill_e Feb 19 '14 at 22:03
  • 1
    $\begingroup$ Sorry my question was unclear, thank you for following up! Yes, the multivariate version of what you linked is what I'm after. Would also appreciate a nudge on how to derive it for myself, not just the formula (that I just can't seem to find). Thank you! $\endgroup$ – bill_e Feb 19 '14 at 22:12
  • 1
    $\begingroup$ Well, if I had to derive a formula I might begin with the generalized least squares version of the univariate case, because the multivariate model can be put into that form (as you indicate after "alternatively"). $\endgroup$ – whuber Feb 19 '14 at 22:18
  • 1
    $\begingroup$ This stuff can get confusing. When I recently encountered such a model, and had to explain it to non-statistical people, I found it useful to spell out the details explicitly for a small example. $\endgroup$ – whuber Feb 19 '14 at 22:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.