1
$\begingroup$

Consider the basic linear model:

$$y_i =\beta_0 +\beta_1 x_i +\varepsilon_i$$

I am aware that if $E[\varepsilon_i|x_i ]=0$ then $E[\hat{\beta_1}]=\beta_1$ (unbiasedness) and also that if $Cov(x_i,\varepsilon_i)=0$ then $plim \hat{\beta_1} = \beta_1$ (consistency).

I am hoping to prove that if $Cov(x_i,\varepsilon_i) \ne 0$ then $E[\hat{\beta_1}]\ne\beta_1$. Can anyone help? My attempt begins below.

$$\hat{\beta_1}=\frac{\sum_{i=1}^n (x_i-\bar{x})(y_i-\bar{y})}{\sum_{i=1}^n (x_i-\bar{x})^2}$$

$$\hat{\beta_1}=\beta_1 + \frac{\sum_{i=1}^n (x_i-\bar{x})(\varepsilon_i-\bar{\varepsilon})}{\sum_{i=1}^n (x_i-\bar{x})^2}$$

$$E[\hat{\beta_1}]=\beta_1 + E\left[\frac{\sum_{i=1}^n (x_i-\bar{x})(\varepsilon_i-\bar{\varepsilon})}{\sum_{i=1}^n (x_i-\bar{x})^2}\right]$$

My goal is to show the expectation on the right must be non-zero as a result of $Cov(x,\varepsilon)\ne 0$. The difficulty is that the expectation operator can be separately applied to the numerator and denominator. The numerator is in expectation non-zero, but if the sample covariance of $x$ and $\varepsilon$ is not independent of the sample variance of $x$, I don't know how to proceed.

Thanks in advance!

$\endgroup$
2
  • $\begingroup$ Consider the expectation in the LHS of the third line of your display. The denominator is a constant and it is non-zero, so can be factored out of the expectation. $E(\sum_{i=1}^n (x_i - \bar{x}) (\epsilon_i - \bar{\epsilon}))$ is by definition the covariance of the $x$ and $\epsilon$ vectors. $\endgroup$
    – AdamO
    Jan 6, 2022 at 18:03
  • $\begingroup$ I am not convinced the denominator is constant, it depends on the draw of data. $\endgroup$ Jan 6, 2022 at 19:33

2 Answers 2

3
$\begingroup$

You are almost there. Remember that $\bar{\epsilon}=0$, you get: $$ \hat{\beta}_1=\beta+\frac{\sum_{i=1}^n(x_i-\bar{x})(\epsilon_i-\bar{\epsilon})}{\sum_{i=1}^n(x_i-\bar{x})^2}=\beta+\frac{\sum_{i=1}^n(x_i-\bar{x})\epsilon_i}{\sum_{i=1}^n(x_i-\bar{x})^2} $$ Therefore $$ E(\hat{\beta}_1 \vert x)=E\left(\left. \beta+\frac{\sum_{i=1}^n(x_i-\bar{x})\epsilon_i}{\sum_{i=1}^n(x_i-\bar{x})^2}\right|x\right)=\beta+\frac{\sum_{i=1}^n(x_i-\bar{x})E(\epsilon_i\vert x)}{\sum_{i=1}^n(x_i-\bar{x})^2}\neq \beta $$ since $E(\epsilon_i \vert x)\neq 0$. Since, $E(\hat{\beta}_1 \vert x) \neq \beta$ it follows that $E(\hat{\beta}_1)=E(E(\hat{\beta}_1 \vert x))\neq \beta$.

$\endgroup$
10
  • $\begingroup$ But how do we know that there is no other $E[\epsilon_i|x]$ (other than $E[\epsilon_i|x]=0$) such that $E[\hat{\beta}|x]=0$? Does this proof definitely confirm no other $E[\epsilon_i|x]$ is possible? $\endgroup$ Jan 6, 2022 at 19:29
  • $\begingroup$ $E[\varepsilon|x]$ is a function, not necessarily a constant number, and I'm just looking for that final step (or, perhaps, missing why that step isn't necessary). $\endgroup$ Jan 6, 2022 at 19:32
  • $\begingroup$ Basically, you need $E(\hat{\beta}_1 \vert x)=\beta$ for $\hat{\beta}_1$ to be consistent. If you look at the expression for $\hat{\beta}_1$, you see that this can only be the case if the last term is equal to zero. Conditional on $x$, the only thing that can make this expression equal to zero is $E(\epsilon_i \vert x)=0$ for all $i$. Thus, if you have one $E(\epsilon_i \vert x) \neq 0$ (no matter what the expression might look like), the result is $E(\hat{\beta}_1 \vert x) \neq \beta$. $\endgroup$
    – Count
    Jan 6, 2022 at 20:01
  • $\begingroup$ But, suppose $E[\hat{\beta_1}|x]\ne \beta$ for some $x$, it may still be that $E[\hat{\beta_1}]=\beta.$ I feel like you have shown sufficiency, but not necessity of the condition $E[\epsilon_i|x]=0$ for unbiasedness. $\endgroup$ Jan 6, 2022 at 20:44
  • $\begingroup$ If $E(\hat{\beta}_1 \vert x)\neq \beta$ it must hold by the law of iterated expectation that $E(\hat{\beta}_1 )=E(E(\hat{\beta}_1\vert x))\neq \beta$. Regarding sufficiency and necessity, do you mean that one can find such $x_1,\dots,x_n$ that, despite the fact that $E(\epsilon_i \vert x)\neq0$, the last term is equal to zero? $\endgroup$
    – Count
    Jan 6, 2022 at 21:03
1
$\begingroup$

Woolridge does a really great job of explaining the unbiasedness of OLS coefficients in his book Introductory Econometrics. In what follows, I use his notation and arguments.

The assumption that $E(u\vert x)=E(u) = 0$ is a central assumption on OLS. This assumption means that the error $u$ and the covariate $x$ are independent. If the error and covariate are not independent (hence correlated), then $E(u\vert x) \neq 0 $ and will in genereal depend on $x$.

The proof that $\hat{\beta}_1$ is an unbiased estimate of $\beta_1$ makes use of this assumption. Following Woolrdige...

$$\hat{\beta}_{1}=\frac{\sum_{i=1}^{n}\left(x_{i}-\bar{x}\right) y_{i}}{\operatorname{SST}_{x}}=\frac{\sum_{i=1}^{n}\left(x_{i}-\bar{x}\right)\left(\beta_{0}+\beta_{1} x_{i}+u_{i}\right)}{\operatorname{SST}_{x}}$$

Here $\mathrm{SST}_{x}=\sum_{i=1}^{n}\left(x_{i}-\bar{x}\right)^{2}$. Some commitment to doing some algebra yields

$$ \hat{\beta}_{1}=\beta_{1}+\left(1 / \operatorname{SST}_{x}\right) \sum_{i=1}^{n} (x_i - \bar{x}) u_{i} $$

If we take expectations in the above expression for $\hat{\beta}_1$, we obtain

$$ E(\hat{\beta}_1) = \beta_1 + 1/(\operatorname{SST}_x)\sum_{i=1}^n (x_i - \bar{x})E(u_i) $$

Note that since the error and predictor are not independent, $E(u_i)$ depends on $x$ and is not 0 in general, this the sum is not 0 leading to a bias.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.