0
$\begingroup$

My question concerns the variance of the mean response as outlined in this short article or in this Wikipedia entry. Basically, the variance of the mean response is given by

$$\text{Var} \left(\hat{\alpha} + \hat{\beta} x_0\right)= \sigma^2 \left(\frac{1}{n} + \frac{(x_0 - \bar{x})^2}{\sum(x_i-\bar{x})^2}\right),$$

where $x_0$ is the data point at which the mean response is predicted.

What I am interested in, however, is a slight modification of this. I would like to compute the variance of the mean response at the mean of the data. If I simply use $x_0 = \bar{x}$, the result is straight forward:

$$\text{Var} \left(\hat{\alpha} + \hat{\beta} \bar{x}\right)= \sigma^2 \left(\frac{1}{n} + \frac{(\bar{x} - \bar{x})^2}{\sum(x_i-\bar{x})^2}\right) = \frac{\sigma^2}{n}$$

But this is not exactly what I am interested in. According to my intuition, since $\bar{x} = \mathrm{E}[x]$ (i.e. $\bar{x}$ is estimated as well), the variance should account for the uncertainty in $\mathrm{E}[x]$. So, I am interested in

$$\text{Var} \left(\hat{\alpha} + \hat{\beta} \mathrm{E}[x]\right)$$

Using basic algebra, I arrive at: $$\text{Var} \left(\hat{\alpha} + \hat{\beta} \mathrm{E}[x]\right) = \mathrm{E}\left[\left(\hat{\alpha} + \hat{\beta} \mathrm{E}[x]\right)^2\right] - \mathrm{E}\left[\hat{\alpha} + \hat{\beta} \mathrm{E}[x]\right]^2 $$ $$ = \mathrm{E}\left[\hat{\alpha}^2\right] - \mathrm{E}\left[\hat{\alpha}\right]^2 +2\mathrm{E}\left[\hat{\alpha}\hat{\beta}\mathrm{E}\left[x\right]\right] -2\mathrm{E}\left[\hat{\alpha}\right]\mathrm{E}\left[\hat{\beta}\right]\mathrm{E}\left[x\right] + \mathrm{E}\left[\hat{\beta}^2\mathrm{E}\left[x\right]^2\right] -\mathrm{E}\left[\hat{\beta}\right]^2\mathrm{E}\left[x\right]^2 $$ In the case where $x_0$ is not stochastic, this simplifies to variances and covariances of the parameters. $\text{Var} \left(\hat{\alpha}\right)$ is still present as the first two terms, but how do I continue simplifying the rest of the equation? Or, does anyone know the result (i.e. how to express $\text{Var} \left(\hat{\alpha} + \hat{\beta} \mathrm{E}[x]\right)$)?

Many thanks for any hint! I am happy to provide further clarification of what I mean if it is unclear.

$\endgroup$
1
$\begingroup$

If defined, there is no uncertainty of $E[X]$ because it is constant, and doesn't depend on data. It's defined purely by the distribution. That is also why $\bar{x}\neq E[X]$, which contradicts your intuition. Sample mean is calculated from data points. So, since you can treat $E[X]$ as if it is another $x_0$, just substitute into your first equation to obtain the variance of the mean response at that point.

Edit regarding your comment: When $\bar{X}$ is stochastic (i.e. $X_i$ are stochastic), what we have is actually $\operatorname{var}(\hat{\alpha}+\hat{\beta}\bar{X}|\bar{X})$, i.e. $\bar{X}$ is known. We'll use Law of Total Variance to find unconditional variance. Note also that $\hat{\alpha},\hat{\beta}$ are unbiased estimators of the true coefficients. $$E[\hat{\alpha}+\hat{\beta}\bar{X}|\bar{X}]=\alpha+\beta\bar{X}$$ Substituting into the law of TV: $$\operatorname{var}(\hat{\alpha}+\hat{\beta}\bar{X})=\operatorname{E\left[\frac{\sigma^2}{n}\right]}+\operatorname{var}(\alpha+\beta\bar{X})=\frac{\sigma^2}{n}+\frac{\beta^2\sigma_x^2}{n}$$

where $\sigma_x^2$ is the variance of $X_i$.

$\endgroup$
  • $\begingroup$ Ok, so basically, it is $\frac{\sigma^2}{n}$? $\endgroup$ – Adrian Muller Jun 21 at 15:25
  • $\begingroup$ No. You'll put $x_0=E[X]$ in the first equation and leave it as it is. You need to convince yourself that $\bar{x}\neq E[X]$ in general, before closing up this issue by the way. $\endgroup$ – gunes Jun 21 at 15:26
  • $\begingroup$ I now see that $\mathbb{E}(x) \neq \bar{x}$, since the expectation is the true mean of the distribution, while the other is the sample mean. If I use the (estimated) sample mean (which has a variance), does this not change the variance of the mean response at all? Is it just like a constant? $\endgroup$ – Adrian Muller Jun 21 at 15:47
  • $\begingroup$ Normally, sample mean has a variance as you said; but here $x_i$'s are also known numbers. So is $\bar{x}$. If they were stochastic, they couldn't get out of the variance expression. This is also noted in the short article you provided (3rd line). $\endgroup$ – gunes Jun 22 at 0:33
  • $\begingroup$ Any hint as to where I would need to look if I do not assume the $x_i$ known? Is there even a way to compute the variance of the forecast if the mean has to be estimated (and hence has a variance)? $\endgroup$ – Adrian Muller Jun 22 at 19:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.