1
$\begingroup$

When is the linear regression estimate of $\beta_1$ in the model

$$ Y= X_1\beta_1 + \delta$$

unbiased, given that the $(x,y)$ pairs are generated with the following model?

$$ Y= X_1\beta_1 + X_2\beta_2 + \delta$$

We have that the expected value of $\beta_1$ is

\begin{align*} E[\hat{\beta}_1|X_1,X_2) &= E[(X_1^TX_1)^{-1}X_1^T(X_1\beta_1+X_2\beta_2+\delta)|X_1,X_2]\\ &=\beta_1 + E[(X_1^TX_1)^{-1}X_1^TX_2\beta_2+(X_1^TX_1)^{-1}X_1^T\delta|X1,X2]\\ &= \beta_1+E[(X_1^TX_1)^{-1}X_1^TX_2\beta_2 | X_1,X_2] + 0\\ \end{align*}

Now, when is the second term 0 (i.e., $\hat{\beta}_1$ is an unbiased estimator)? I have read that it is 0 if $X_1$ and $X_2$ are independent.

But which property allows me to conclude that?

$\endgroup$
  • $\begingroup$ I don't see any justification for the isolated appearance of $\beta_1$ in the second line of the calculation. In fact, the first line does not appear to be the correct formula for the least squares estimate, because it ignores the presence of $\delta$ in the model. $\endgroup$ – whuber Feb 21 '16 at 20:04
  • $\begingroup$ Isn't delta the error? $\endgroup$ – JohnK Feb 21 '16 at 20:05
  • $\begingroup$ Yes, delta is the error $\endgroup$ – JC1 Feb 21 '16 at 20:10
  • 1
    $\begingroup$ @whuber I believe the idea is that while it is the second model that is the correct one, we carelessly estimate the first one, the reduced model. Hence the LS estimate based on the $\mathbf{X}_1$ matrix. The point is to demonstrate the bias of the estimator in that case. Of course, it's better if the OP confirms that. $\endgroup$ – JohnK Feb 21 '16 at 20:13
  • $\begingroup$ Yes, you're right John $\endgroup$ – JC1 Feb 21 '16 at 20:19
1
$\begingroup$

It is zero when the columns of $\mathbf{X}_1$ are perpendicular to the columns of $\mathbf{X}_2$ so that the column spaces are orthogonal to one another. This means that the variables need to be uncorrelated to one another, which is not quite the same thing as independence. It is in fact a weaker condition as independence implies zero correlation.

If the variables are not uncorrelated, however, and you proceed to estimate $\boldsymbol{\beta}_1$ only, you will end up with a biased estimator. In fact, this is called omitted variable bias.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.