0
$\begingroup$

I am wondering if:

  1. omitting an important variable

and

  1. having very high correlation between two independent variables

causes the OLS estimators to become biased. If so, can you please provide a short explanation?

$\endgroup$
  • 4
    $\begingroup$ Correlated independent variables do not cause the OLS estimators to be biased, they will only have higher variance. Omitting a relevant dependent variable that is correlated with the remaining dependent variables, biases the coefficients. So only by omitting a variable you introduce bias, that bias will be higher if the omitted variable is strongly correlated with the remaining dependent variables. $\endgroup$ – user83346 Oct 22 '16 at 14:01
  • $\begingroup$ Hello, thank you for the answer. So, if the omitted variable is correlated with the other dependent variables, the OLS estimators becomes in general biased. Can you expand a bit on that? $\endgroup$ – user358065 Oct 22 '16 at 14:07
  • $\begingroup$ Sorry, it should say ''independent variables''. $\endgroup$ – user83346 Oct 22 '16 at 14:11
  • 3
    $\begingroup$ Assume that you model $y=\beta_0 + \beta_1 x_1 + \beta_2 x_2 + \epsilon$ where $x_1$ and $x_2$ are correlated. Then $x_1$ and $x_2$ ''move together'', so if you now drop one f the two e.g. $x_1$, then part of the effect (i.e. the coefficient) of $x_1$ will be ''captured'' in the coefficient of $x_2$, biasing latter $\endgroup$ – user83346 Oct 22 '16 at 14:15
  • $\begingroup$ There are a couple of answers on here related to omitted variable bias (i.e. omitting an important variable from the analysis which is correlated with independent variables in your model). For a more detailed answer, see: stats.stackexchange.com/questions/315901/… $\endgroup$ – Wes Dec 18 '17 at 12:47
7
$\begingroup$

(1) Yes, leaving out a regressor can introduce bias into your regression, under certain conditions.

That is, if $Y = \beta_0 + \beta_1 X_1 + \beta_2 X_2 + U$ is the true data generating process, and you estimated the regression using just $X_1$, then it can be shown that

$$ \hat \beta_1^\ast \overset{p}{\to} \beta_1 + \beta_2\frac{Cov(X_1,X_2)}{Var(X_1)}.$$

Thus, $\hat \beta_1^\ast$ will be biased only if $\beta_2\neq 0$ and $X_1$ and $X_2$ are correlated.

(2) Collinearity will not bias your regression. However, if can make your regression hard/impossible to meaningfully estimate.

In the extreme case, if you have perfect correlation between two variables, then you cannot even estimate the regression, because you will not be able to take the inverse of $(X'X)$ when calculating $\hat \beta = (X'X)^{-1}X'Y$.

However, if you have a very high level of correlation between two variables $X_1$ and $X_2$, then the variance of your two estimates $\beta_1,\beta_2$ will be high. If you think about it intuitively, the regression model doesn't really know whether to assign the effect of increasing the variables (which move together) to $\beta_1$ or $\beta_2$. Thus, it becomes hard to show statistical significance of coefficients, and estimates can be very unstable.

Furthermore, the values in the estimated regression may not be interpretable. Typically, we interpret coefficients as saying "all else equal, a one unit increase in $X_1$ will results in ....". However, a one unit increase in $X_1$ all else equal will never happen, because $X_1$ and $X_2$ are so correlated.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.