4
$\begingroup$

The Gauss-Markov theorem states that for a linear model

$$y = X \beta + \epsilon $$

if both of the conditions are true

$$\operatorname E[\epsilon \mid X] = 0$$ $$\operatorname{Var}(\epsilon) = \sigma^2 I < \infty $$

then the standard OLS estimator $(X'X)^{-1}X'y$ is the best linear unbiased estimator.

Now suppose we measure $X$ with errors. Then we have

$$y = (X + \mu)\beta + \epsilon = X\beta + \mu\beta+\epsilon$$

If $\mu$ is of mean $0$ with constant variance, both assumptions still hold. Why then is the OLS estimator biased?

$\endgroup$
5
  • 3
    $\begingroup$ I would say the OLS estimator is $(X'X)^{-1}X'y,$ not just $(X'X)^{-1}X'.$ In particular, the form $(X'X)^{-1}X'y$ shows you why the word "linear" is used: It's linear as a function of $y. \qquad$ $\endgroup$ Nov 15, 2018 at 16:54
  • 3
    $\begingroup$ $(X'X)^{-1} X' y$ is still the best linear unbiased estimator. $((X+\mu)'(X+\mu))^{-1}(X+\mu)' y$ is not. $\endgroup$
    – Frank
    Nov 15, 2018 at 17:23
  • 2
    $\begingroup$ Let $z = x + u$. If you rewrite in terms of observable $z$, your problem is $\operatorname{E}[u \mid z] \neq 0$ $\endgroup$ Nov 15, 2018 at 18:16
  • $\begingroup$ Is this a homework question? $\endgroup$ Nov 15, 2018 at 20:16
  • $\begingroup$ No, I thought that noise in the independent variables shouldn't bias the coefficients if it's independent but everything I've read online told me otherwise. Couldn't figure out why. $\endgroup$
    – badmax
    Nov 15, 2018 at 20:52

1 Answer 1

2
$\begingroup$

Let's say the true data generating process is: $$ y_i = x_i \beta + \epsilon_i $$ But we don't observe $x_i$, instead we observe $ z_i = x_i + u_i$. We can write the above using observables ($z_i, y_i$): $$ y_i = z_i \beta + v_i $$ Where the error term is $ v_i = \epsilon_i - \beta u_i$. Is the strict exogeneity requirement $\operatorname{E}[v \mid z] = 0$ satisfied? No.

  • If $\operatorname{E}[v \mid z] = 0$ then $\operatorname{E}[vz]=0$, but $\operatorname{E}[vz]=\operatorname{E}[(\epsilon - \beta u)(x + u)] = - \beta \operatorname{E}[u^2] $. $\bot$

The underlying cause is that $\operatorname{E}[u \mid x + u] \neq 0$. The precise story depends on the distribution of $x$ and $u$, but loosely speaking, above average measurements $z$ are going to be associated with positive measurement error $u$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.