Proof OLS is Biased if a Regressor is Correlated with the error

Question

Consider the basic linear model:

$$y_i =\beta_0 +\beta_1 x_i +\varepsilon_i$$

I am aware that if $E[\varepsilon_i|x_i ]=0$ then $E[\hat{\beta_1}]=\beta_1$ (unbiasedness) and also that if $Cov(x_i,\varepsilon_i)=0$ then $plim \hat{\beta_1} = \beta_1$ (consistency).

I am hoping to prove that if $Cov(x_i,\varepsilon_i) \ne 0$ then $E[\hat{\beta_1}]\ne\beta_1$. Can anyone help? My attempt begins below.

$$\hat{\beta_1}=\frac{\sum_{i=1}^n (x_i-\bar{x})(y_i-\bar{y})}{\sum_{i=1}^n (x_i-\bar{x})^2}$$

$$\hat{\beta_1}=\beta_1 + \frac{\sum_{i=1}^n (x_i-\bar{x})(\varepsilon_i-\bar{\varepsilon})}{\sum_{i=1}^n (x_i-\bar{x})^2}$$

$$E[\hat{\beta_1}]=\beta_1 + E\left[\frac{\sum_{i=1}^n (x_i-\bar{x})(\varepsilon_i-\bar{\varepsilon})}{\sum_{i=1}^n (x_i-\bar{x})^2}\right]$$

My goal is to show the expectation on the right must be non-zero as a result of $Cov(x,\varepsilon)\ne 0$. The difficulty is that the expectation operator can be separately applied to the numerator and denominator. The numerator is in expectation non-zero, but if the sample covariance of $x$ and $\varepsilon$ is not independent of the sample variance of $x$, I don't know how to proceed.

Thanks in advance!

Answer 1

You are almost there. Remember that $\bar{\epsilon}=0$, you get: $$ \hat{\beta}_1=\beta+\frac{\sum_{i=1}^n(x_i-\bar{x})(\epsilon_i-\bar{\epsilon})}{\sum_{i=1}^n(x_i-\bar{x})^2}=\beta+\frac{\sum_{i=1}^n(x_i-\bar{x})\epsilon_i}{\sum_{i=1}^n(x_i-\bar{x})^2} $$ Therefore $$ E(\hat{\beta}_1 \vert x)=E\left(\left. \beta+\frac{\sum_{i=1}^n(x_i-\bar{x})\epsilon_i}{\sum_{i=1}^n(x_i-\bar{x})^2}\right|x\right)=\beta+\frac{\sum_{i=1}^n(x_i-\bar{x})E(\epsilon_i\vert x)}{\sum_{i=1}^n(x_i-\bar{x})^2}\neq \beta $$ since $E(\epsilon_i \vert x)\neq 0$. Since, $E(\hat{\beta}_1 \vert x) \neq \beta$ it follows that $E(\hat{\beta}_1)=E(E(\hat{\beta}_1 \vert x))\neq \beta$.

Answer 2

Woolridge does a really great job of explaining the unbiasedness of OLS coefficients in his book Introductory Econometrics. In what follows, I use his notation and arguments.

The assumption that $E(u\vert x)=E(u) = 0$ is a central assumption on OLS. This assumption means that the error $u$ and the covariate $x$ are independent. If the error and covariate are not independent (hence correlated), then $E(u\vert x) \neq 0 $ and will in genereal depend on $x$.

The proof that $\hat{\beta}_1$ is an unbiased estimate of $\beta_1$ makes use of this assumption. Following Woolrdige...

$$\hat{\beta}_{1}=\frac{\sum_{i=1}^{n}\left(x_{i}-\bar{x}\right) y_{i}}{\operatorname{SST}_{x}}=\frac{\sum_{i=1}^{n}\left(x_{i}-\bar{x}\right)\left(\beta_{0}+\beta_{1} x_{i}+u_{i}\right)}{\operatorname{SST}_{x}}$$

Here $\mathrm{SST}_{x}=\sum_{i=1}^{n}\left(x_{i}-\bar{x}\right)^{2}$. Some commitment to doing some algebra yields

$$ \hat{\beta}_{1}=\beta_{1}+\left(1 / \operatorname{SST}_{x}\right) \sum_{i=1}^{n} (x_i - \bar{x}) u_{i} $$

If we take expectations in the above expression for $\hat{\beta}_1$, we obtain

$$ E(\hat{\beta}_1) = \beta_1 + 1/(\operatorname{SST}_x)\sum_{i=1}^n (x_i - \bar{x})E(u_i) $$

Note that since the error and predictor are not independent, $E(u_i)$ depends on $x$ and is not 0 in general, this the sum is not 0 leading to a bias.