# OLS Regression with Endogenous Variables

I know that endogenous variables will result in inconsistent OLS estimates, but what do these estimates actually turn out to be if we try to use an OLS linear regression? Anywhere I look is just simply telling me not to do it, but I'd be interested in seeing the relationship which does arise.

Say, for example, we have:

Where is not exogenous, though is.

If we attempted to estimate the parameters by OLS, what estimates would we get, and how would these relate to the inconsistency arising from the endogeneity?

• There is a derivation of the bias term that persists in large samples (hence, inconsistency) on slides 10-13 here: uio.no/studier/emner/sv/oekonomi/ECON4150/v17/…. It addresses the case when there are no other predictors in the model besides the endogenous variable. Are you specifically interested in the case when there is another variable included ($X_1$ in your case)? That will require a more complicated answer, as it will depend on possible correlation between included predictors. – AlexK Jun 3 '19 at 7:58

This is very general question, so I will try to make general sketch. Endogeneity in narrow sense is violation of condition $$cov(X,u)=0$$, in other words, in your example one of regressors $$X_2$$ is correlated with error term $$u$$.

All other things being fine (true model is linear, not other omitted variables etc.), we expect that OLS in your example will return us biased estimation $$b_2$$ of coefficient $$\beta_2$$. Biased in simple terms means that we systematically get wrong (shifted) value of the coefficient in question.

Notice that books warn us not to use such models if we are interested in relationship between $$Y$$ and $$X_2$$. But if we are interested, say, in $$\beta_1$$ only, it might be fine, and $$X_2$$ is regarded as control variable in such cases (though, it doesn't mean that control variables must be endogenous).

Another class of cases where your model can be acceptable is if we what to simply predict some values of $$Y$$ without need to explain (or understand) causal relationships in the model. It it works for such a purpose, why not.

It is boring statement, but always choice of a model depends on real needs of the modeler.

Small numerical example I will use Mathematica code and try to comment it.

Cook variables (for 20 observations).

SeedRandom; u = RandomReal[{-1, 1}, 20];
x1 = Range;
x2 = Range[20, 1, -1] (u);

y = 5 - 1 x1 + 3 x2 + 6 u


Let us take a look at correlations between regressors and errors numerically and visually.

{Correlation[x1, x2], Correlation[x1, u], Correlation[x2, u]}


{0.295826, 0.182349, 0.866208} It is fine to have some covariance between $$X_1$$ and $$X_2$$ in multiple regression models; the main thing here to avoid perfect colinearity.

Our OLS is as follows:

lm = LinearModelFit[Thread[{x1, x2, y}], {X1, X2}, {X1, X2}];
lm["BestFit"]


5.79933 - 1.05241 X1 + 3.45583 X2

You may see that we've got $$b_2 = 3.45$$ instead of expected $$3$$. How fine is that?

If we fit a model without endogeneity 1000 times we may build such histograms of coefficients (for $$b_1$$ to the left and $$b_2$$ to the right). You may see that it is highly unlikely to get the point estimate that we have obtained from our model. We have got inconsistent estimate in the sense that $$b_2$$ does not approach to $$\beta_2$$ even if we had more data.

Nonetheless, we cannot get rid of $$X_2$$ to evaluate $$b_1$$ (because of some covariance between regressors). Notice that $$b_1$$ looks just fine.

You may also want to see geometry of endogeneity in simple OLS here.