Consider the linear regression model: $$y_{it}=x_{it}\beta+\epsilon_{it}$$ where $x$ is single regressor. The conditional mean of any specific observation is:$$E[y_{it}|x_{it}]=x_{it}\beta$$ under the conditional mean independence assumption. The conditional variance, howevever, is where I am confused. On one had, we can take the variance of both sides of the original equation and get:$$var(y_{it})=\beta^{2}var(x_{it})+var(\epsilon_{it})+2cov(x_{it}\beta,\epsilon_{it})=\beta^{2}\sigma_{x}^{2}+\sigma_{\epsilon}^{2}+0$$

On the other hand, think of the following formula for the variance:$$E\left[y_{it}-E\{y_{it}\}\right]^2 =E[\epsilon_{it}^{2}]=\sigma_{\epsilon}^{2}$$

Which one is correct?


The title of your question "Conditional variance in OLS regression" gives a clue. The first expression


gives the unconditional variance ($x$ is not "conditioned away" and remains in the expression) while the second one

$$\text{E}\left[y_{it}-\text{E}\{y_{it}\}\right]^2 =\sigma_{\epsilon}^{2}$$

gives the conditional variance (conditional on $x$).

| cite | improve this answer | |

But your final equation cannot be right if you are not treating the predictors as fixed. For

$$y_i = \beta x_i + \epsilon_i$$

recalling that

$$E\left(Y \right) = E\left[ E \left(Y|X \right) \right]$$

we have

$$E(y_i) =E\left( E\left(\beta x_i + \epsilon_i|x_i \right) \right) = \beta \mu_x$$

and so

\begin{align} Var(y_i) = E\left(y_i ^2\right) - \left(E(y_i) \right)^2 &= E \left(\beta x_i + \epsilon_i \right)^2 - \beta^2 \mu_x^2 \\ &= \beta^2 E\left(x_i^2 \right) + E(\epsilon^2_i) + 2 E\left( \beta x_i \epsilon_i \right) - \beta^2 \mu_x^2 \\ & = \beta^2 \left(\sigma_x^2 + \mu^2_x \right) + \sigma^2 -\beta^2 \mu_x^2 \\ & = \beta^2 \sigma^2_x + \sigma^2 \end{align}

assuming that the $x_i$ are iid. This is similar to what you obtained in your third equation of course.

| cite | improve this answer | |
  • 2
    $\begingroup$ Meaning also that if you ARE treating your predictors as fixed (which is easier and common in introductory classes), both equations are correct, being equal as sigma-x=0. $\endgroup$ – one_observation Jan 13 '16 at 3:56
  • 1
    $\begingroup$ @Sophologist Yes, did I contradict that in any way? $\endgroup$ – JohnK Jan 13 '16 at 9:57
  • 1
    $\begingroup$ Not at all - I just wanted to be explicit about the multiple consequences of that assumption, given that the confusion appears to stem from not having considered it in the first place. $\endgroup$ – one_observation Jan 13 '16 at 12:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.