# Error distributions and consistent and unbiased OLS

If OLS estimator is unbiased and consistent, what does it imply about the distribution of error terms?

In linear regression model: $$y_i = \boldsymbol{x_i' \beta} + \epsilon_i$$ if the OLS estimator of $$\boldsymbol{\beta}$$ is unbiased and consistent, what are the implications for the distribution of $$\epsilon_i$$? Mainly, does it imply that the distribution is symmetric?

• No, symmetry is not required, only that $\mathbb{E}\epsilon_i = 0$ and that $X$ and $\epsilon$ are independent (actually, I believe the latter is stronger than required.) A finite variance is not required, but of course the existence of the expectation is. – jbowman Nov 23 '19 at 15:49

Let's write out the expression for $$\hat{\beta}_{OLS}$$:

$$\hat{\beta}_{OLS} = (X^TX)^{-1}X^Ty = (X^TX)^{-1}X^T(X\beta+\epsilon) = \beta+(X^TX)^{-1}X^T\epsilon$$

Therefore

$$\mathbb{E}\hat{\beta}_{OLS} = \mathbb{E}[\beta+(X^TX)^{-1}X^T\epsilon]$$

If $$X$$ and $$\epsilon$$ are independent, $$\mathbb{E}(X^TX)^{-1}X^T\epsilon = (X^TX)^{-1}X^T\mathbb{E}\epsilon$$, so, rewriting,

$$\mathbb{E}\hat{\beta}_{OLS} = \beta + (X^TX)^{-1}X^T\mathbb{E}\epsilon$$

and all that is left for unbiasedness is that $$\mathbb{E}\epsilon = 0$$.

This can obviously be weakened somewhat by noting that as long as $$\mathbb{E}(X^TX)^{-1}X^T\epsilon = 0$$, we will have unbiasedness, but the assumption that this holds when $$\mathbb{E}\epsilon = 0$$ does not is typically hard to check or justify, although it can occur.

Symmetry does not appear anywhere in this derivation, nor does the existence or finiteness of of $$\sigma^2_{\epsilon}$$.

• Math is undeniable. But I was conceptually expecting a symmetric distribution because the average square error function is symmetric. I think it is a happy case of linearity. – Cagdas Ozgenc Nov 23 '19 at 18:27