# Proof OLS is Biased if a Regressor is Correlated with the error

Consider the basic linear model:

$$y_i =\beta_0 +\beta_1 x_i +\varepsilon_i$$

I am aware that if $$E[\varepsilon_i|x_i ]=0$$ then $$E[\hat{\beta_1}]=\beta_1$$ (unbiasedness) and also that if $$Cov(x_i,\varepsilon_i)=0$$ then $$plim \hat{\beta_1} = \beta_1$$ (consistency).

I am hoping to prove that if $$Cov(x_i,\varepsilon_i) \ne 0$$ then $$E[\hat{\beta_1}]\ne\beta_1$$. Can anyone help? My attempt begins below.

$$\hat{\beta_1}=\frac{\sum_{i=1}^n (x_i-\bar{x})(y_i-\bar{y})}{\sum_{i=1}^n (x_i-\bar{x})^2}$$

$$\hat{\beta_1}=\beta_1 + \frac{\sum_{i=1}^n (x_i-\bar{x})(\varepsilon_i-\bar{\varepsilon})}{\sum_{i=1}^n (x_i-\bar{x})^2}$$

$$E[\hat{\beta_1}]=\beta_1 + E\left[\frac{\sum_{i=1}^n (x_i-\bar{x})(\varepsilon_i-\bar{\varepsilon})}{\sum_{i=1}^n (x_i-\bar{x})^2}\right]$$

My goal is to show the expectation on the right must be non-zero as a result of $$Cov(x,\varepsilon)\ne 0$$. The difficulty is that the expectation operator can be separately applied to the numerator and denominator. The numerator is in expectation non-zero, but if the sample covariance of $$x$$ and $$\varepsilon$$ is not independent of the sample variance of $$x$$, I don't know how to proceed.

• Consider the expectation in the LHS of the third line of your display. The denominator is a constant and it is non-zero, so can be factored out of the expectation. $E(\sum_{i=1}^n (x_i - \bar{x}) (\epsilon_i - \bar{\epsilon}))$ is by definition the covariance of the $x$ and $\epsilon$ vectors. Jan 6 at 18:03
• I am not convinced the denominator is constant, it depends on the draw of data. Jan 6 at 19:33

You are almost there. Remember that $$\bar{\epsilon}=0$$, you get: $$\hat{\beta}_1=\beta+\frac{\sum_{i=1}^n(x_i-\bar{x})(\epsilon_i-\bar{\epsilon})}{\sum_{i=1}^n(x_i-\bar{x})^2}=\beta+\frac{\sum_{i=1}^n(x_i-\bar{x})\epsilon_i}{\sum_{i=1}^n(x_i-\bar{x})^2}$$ Therefore $$E(\hat{\beta}_1 \vert x)=E\left(\left. \beta+\frac{\sum_{i=1}^n(x_i-\bar{x})\epsilon_i}{\sum_{i=1}^n(x_i-\bar{x})^2}\right|x\right)=\beta+\frac{\sum_{i=1}^n(x_i-\bar{x})E(\epsilon_i\vert x)}{\sum_{i=1}^n(x_i-\bar{x})^2}\neq \beta$$ since $$E(\epsilon_i \vert x)\neq 0$$. Since, $$E(\hat{\beta}_1 \vert x) \neq \beta$$ it follows that $$E(\hat{\beta}_1)=E(E(\hat{\beta}_1 \vert x))\neq \beta$$.

• But how do we know that there is no other $E[\epsilon_i|x]$ (other than $E[\epsilon_i|x]=0$) such that $E[\hat{\beta}|x]=0$? Does this proof definitely confirm no other $E[\epsilon_i|x]$ is possible? Jan 6 at 19:29
• $E[\varepsilon|x]$ is a function, not necessarily a constant number, and I'm just looking for that final step (or, perhaps, missing why that step isn't necessary). Jan 6 at 19:32
• Basically, you need $E(\hat{\beta}_1 \vert x)=\beta$ for $\hat{\beta}_1$ to be consistent. If you look at the expression for $\hat{\beta}_1$, you see that this can only be the case if the last term is equal to zero. Conditional on $x$, the only thing that can make this expression equal to zero is $E(\epsilon_i \vert x)=0$ for all $i$. Thus, if you have one $E(\epsilon_i \vert x) \neq 0$ (no matter what the expression might look like), the result is $E(\hat{\beta}_1 \vert x) \neq \beta$.
– Lars
Jan 6 at 20:01
• But, suppose $E[\hat{\beta_1}|x]\ne \beta$ for some $x$, it may still be that $E[\hat{\beta_1}]=\beta.$ I feel like you have shown sufficiency, but not necessity of the condition $E[\epsilon_i|x]=0$ for unbiasedness. Jan 6 at 20:44
• If $E(\hat{\beta}_1 \vert x)\neq \beta$ it must hold by the law of iterated expectation that $E(\hat{\beta}_1 )=E(E(\hat{\beta}_1\vert x))\neq \beta$. Regarding sufficiency and necessity, do you mean that one can find such $x_1,\dots,x_n$ that, despite the fact that $E(\epsilon_i \vert x)\neq0$, the last term is equal to zero?
– Lars
Jan 6 at 21:03

Woolridge does a really great job of explaining the unbiasedness of OLS coefficients in his book Introductory Econometrics. In what follows, I use his notation and arguments.

The assumption that $$E(u\vert x)=E(u) = 0$$ is a central assumption on OLS. This assumption means that the error $$u$$ and the covariate $$x$$ are independent. If the error and covariate are not independent (hence correlated), then $$E(u\vert x) \neq 0$$ and will in genereal depend on $$x$$.

The proof that $$\hat{\beta}_1$$ is an unbiased estimate of $$\beta_1$$ makes use of this assumption. Following Woolrdige...

$$\hat{\beta}_{1}=\frac{\sum_{i=1}^{n}\left(x_{i}-\bar{x}\right) y_{i}}{\operatorname{SST}_{x}}=\frac{\sum_{i=1}^{n}\left(x_{i}-\bar{x}\right)\left(\beta_{0}+\beta_{1} x_{i}+u_{i}\right)}{\operatorname{SST}_{x}}$$

Here $$\mathrm{SST}_{x}=\sum_{i=1}^{n}\left(x_{i}-\bar{x}\right)^{2}$$. Some commitment to doing some algebra yields

$$\hat{\beta}_{1}=\beta_{1}+\left(1 / \operatorname{SST}_{x}\right) \sum_{i=1}^{n} (x_i - \bar{x}) u_{i}$$

If we take expectations in the above expression for $$\hat{\beta}_1$$, we obtain

$$E(\hat{\beta}_1) = \beta_1 + 1/(\operatorname{SST}_x)\sum_{i=1}^n (x_i - \bar{x})E(u_i)$$

Note that since the error and predictor are not independent, $$E(u_i)$$ depends on $$x$$ and is not 0 in general, this the sum is not 0 leading to a bias.