The question is the following:

  1. Our regression model can be written as $y_i = \beta^Tx_i + \epsilon_i, 1 \leq i \leq n$. Find the $100(1- \alpha)\%$ confidence interval for the mean response $\beta^Tx$ at a given vector $x$ of regressor observations.

  2. Now suppose a future observation y is taken at a regressor $x$. Find a $100(1-\alpha)\% $ prediction interval for $y$.


  • $k$ denote the # of regressors
  • $n$ denotes the sample size
  • $\sigma^2$ is ${\rm Var}(\epsilon|X)$ where $X$ is the $nxk$ matrix containing all the regressor values.
  • $x_i$ is a $nx1$ vector of all the sampled values of the $i^{th}$ regressor

So from my understanding the mean response is $E[y_i|x_i]$ where $1 \leq j \leq n$. And it is obvious that $E[y_i|x_i] = \beta^Tx$. However, I am unsure of how to get the variance needed to get the confidence interval for #1. Using $y_i \pm t_{\alpha/2, n-k-1}\sigma_{y_i}$.

I used the following method as an attempt:

$${\rm Var}(\beta^Tx_i|x_i) = x^T_i {\rm Var}(\beta|x_i) x_i$$

I took ${\rm Var}(\beta|x_i) = \sigma^2 (x^Tx)^{-1}$, which is the value ${\rm Var}(\beta_{OLS}|X)$, where $X$ is the $nxk$ matrix containing all the regressor values.

So I got the answer:

$$y_i \pm t_{\alpha/2, n-k}\sqrt{x_i^T\sigma^2(x^Tx)^{-1}x_i}$$

which is little more than a horrible guess since for a mean response only $x_i$ is given.

I tried doing it this way too:
$${\rm Var}(\Sigma_{j=1}^k \beta_jx_{ij}|x_i)$$ but the expansion was too ugly to bear...

For #2, I got the following answer: $$ \hat y_{OLS} \pm t_{\alpha/2, n-k}\sqrt{\frac{e^Te}{n-k}} $$ which I am afraid another guess... where I assumed it was just an estimate using Ordinary Least Square method.

Please point to a method of solving this variance problem. Please explain your rational!

  • $\begingroup$ Thank you for showing what you've tried so far & where you are stuck. Our policy is to provide hints to help get you unstuck & to the point where you can do this on your own. For more info, you may want to read the wiki for the self-study tag. $\endgroup$ – gung Aug 11 '14 at 0:08
  • $\begingroup$ Please check the first line of your quoted question. Is there something missing after $\beta^T$? $\endgroup$ – Glen_b Aug 11 '14 at 0:31
  • $\begingroup$ What's your estimate of $\beta^Tx$? Can you use basic properties of variances or expectations to compute its variance? $\endgroup$ – Glen_b Aug 11 '14 at 0:33
  • $\begingroup$ This is a theoretical question. There are no concrete values. In my post, I talked about computing $Var(\Sigma_{j=1}^k\beta_jx_{ij}|x_i)$ directly, but it is difficult because the $\beta$ can be correlated which creates one big mess. $\endgroup$ – itzjustricky Aug 11 '14 at 1:00
  • $\begingroup$ I edited my post so that it specifies that $x_i$ are nx1 vectors $\endgroup$ – itzjustricky Aug 11 '14 at 1:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.