0
$\begingroup$

If OLS estimator is unbiased and consistent, what does it imply about the distribution of error terms?

In linear regression model: $ y_i = \boldsymbol{x_i' \beta} + \epsilon_i $ if the OLS estimator of $\boldsymbol{\beta}$ is unbiased and consistent, what are the implications for the distribution of $\epsilon_i$? Mainly, does it imply that the distribution is symmetric?

$\endgroup$
  • $\begingroup$ No, symmetry is not required, only that $\mathbb{E}\epsilon_i = 0$ and that $X$ and $\epsilon$ are independent (actually, I believe the latter is stronger than required.) A finite variance is not required, but of course the existence of the expectation is. $\endgroup$ – jbowman Nov 23 '19 at 15:49
1
$\begingroup$

Let's write out the expression for $\hat{\beta}_{OLS}$:

$$\hat{\beta}_{OLS} = (X^TX)^{-1}X^Ty = (X^TX)^{-1}X^T(X\beta+\epsilon) = \beta+(X^TX)^{-1}X^T\epsilon$$

Therefore

$$\mathbb{E}\hat{\beta}_{OLS} = \mathbb{E}[\beta+(X^TX)^{-1}X^T\epsilon]$$

If $X$ and $\epsilon$ are independent, $\mathbb{E}(X^TX)^{-1}X^T\epsilon = (X^TX)^{-1}X^T\mathbb{E}\epsilon$, so, rewriting,

$$\mathbb{E}\hat{\beta}_{OLS} = \beta + (X^TX)^{-1}X^T\mathbb{E}\epsilon$$

and all that is left for unbiasedness is that $\mathbb{E}\epsilon = 0$.

This can obviously be weakened somewhat by noting that as long as $\mathbb{E}(X^TX)^{-1}X^T\epsilon = 0$, we will have unbiasedness, but the assumption that this holds when $\mathbb{E}\epsilon = 0$ does not is typically hard to check or justify, although it can occur.

Symmetry does not appear anywhere in this derivation, nor does the existence or finiteness of of $\sigma^2_{\epsilon}$.

$\endgroup$
  • $\begingroup$ Math is undeniable. But I was conceptually expecting a symmetric distribution because the average square error function is symmetric. I think it is a happy case of linearity. $\endgroup$ – Cagdas Ozgenc Nov 23 '19 at 18:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.