1
$\begingroup$

I am currently testing my linear model using OLS method. The last thing I have to test is endogeneity issue. Is it enough if I test each explanatory variable for correletion with error term? Than means I save the residuals of my original model and I use them in cor.test in R paired with each explanatory variable? I would like to test first whether there is existence of the endogeneity issue before going further with advanced methods to deal with the problem.

I know there is proper way of testing using for example Hausman test which compares the results of OLS and 2SLS, but using 2SLS and IV seems to be very complicated to me considering the level of my knowledge.

$\endgroup$
1
$\begingroup$

The reply of Dimitry can be enough, as markowitz says, but I'd like to add a very simple simulation:

> set.seed(1234)             
> x <- rnorm(1000)          # predictor
> u <- x + rnorm(1000)      # "true" error, correlated with x
> y <- 3 + 2*x + u          # outcome

Let's fit a linear model:

> fit <- lm(y ~ x)
> fit
[...]
Coefficients:
(Intercept)            x  
      3.029        3.016  

As you can see, the estimated coefficient for $x$ is biased. Why? Because $x$ and $u$ are correlated:

> cor(x,u)
[1] 0.7073596

What about residuals?

> r <- fit$residuals
> cor(x,r)
[1] 2.200033e-17

$x$ and residuals are not correlated, and they are never correlated. Why? Well, we need a bit of math: $$\text{if}\quad\hat\beta=(X^TX)^{-1}X^Ty,\quad\text{then}\quad r=y-X\hat\beta=y-X(X^TX)^{-1}X^Ty$$ and we always have: $$X^Tr=X^Ty-X^TX(X^TX)^{-1}X^Ty=0$$

markovitz says: "I suppose that sabiste conflated the role of residuals with that of true error terms. Common mistake among neophyte." Sure, but not only among neophytes :)

Fifteen years ago a paper argued that "exogeneity constraints that are commonly assumed in econometric treatments of the Gauss-Markov theorem are unnecessary for OLS estimates of the classical linear regression model to be BLU" [...] "because orthogonality is a property of all OLS estimates. The geometry of least squares forces the errors in a regression equation to be orthogonal to all of the regressors in the equation."

A few years later, another paper was published in the same journal. Its title was: Wouldn't It Be Nice...? The Automatic Unbiasedness of OLS (and GLS): "the intrinsic orthogonality he is thinking of is of $X$ with $\hat{u}$ [my $r$], not $u$."

I think that reading those papers could be an (amusing, and) useful way to better understand the endogeneity issue.

$\endgroup$
6
  • $\begingroup$ Amusing articles … more time go on more seem me that confusions are wider and deeper. I changed a bit the definition of error term in my reply in order to taking better into account the classical examples. Let me know if you have some observations. $\endgroup$
    – markowitz
    Jul 22 '20 at 9:30
  • $\begingroup$ @markowitz According to Golberger (A Course in Econometrics, 1991, p. 170) the error vector "is simply the disturbance vector, the deviation of the random vector $y$ from its expectation." According to Wooldridge (Econometric Analysis, 2002, p. 49), "the error term $u$ consist of a variety of things, including omitted variables and measurement error." According to Hayashi (Econometrics, 2000, p. 4), "the error term represents the part of the dependent variable left unexplained by the regressors." It' a hard choice :) $\endgroup$
    – Sergio
    Jul 22 '20 at 10:45
  • $\begingroup$ Unfortunately the interpretation of “error term” is not a detail in econometric models, rather is an pillar of them. If Authors disagree about error term meaning are disagree about the entire meaning of the “true model”, at least if them refers on it (as frequently made). So, them can be disagree about almost all crucial things. Any kind of misunderstanding can emerge from loss of clarity here. Unfortunately this seems me the reality today yet. Several my interventions in this site are related to this point. $\endgroup$
    – markowitz
    Jul 22 '20 at 14:01
  • $\begingroup$ In my view the true model is something like a “structural model”, definitely not a regression! Therefore, the included error term is a structural error (see here: stats.stackexchange.com/questions/455373/…). Note that in your example/simulation you followed the same paradigm. My explanation below skip this problems but is consistent with my view. $\endgroup$
    – markowitz
    Jul 22 '20 at 14:01
  • $\begingroup$ Then, the definition given in Golberger (1991) maintain his statistical respectability but is almost useless in econometrics theory. More precisely, under his definition of error term, the usual definition of exogeneity, the core of econometrics, become a tautology. The others two position that you cited sound like ambiguous. Unfortunately this is a big problem in econometrics today. Position like mine can seems presumptuous, however is not from my own only. $\endgroup$
    – markowitz
    Jul 22 '20 at 14:01
0
$\begingroup$

This would not give you a valid test of endogeneity. Estimated residuals will be uncorrelated with included regressors by construction. You can work through the math or find a derivation, but you can also easily convince yourself of this with a simple simulation.

$\endgroup$
0
$\begingroup$

The reply of Dimitry can be enough. However I suppose that your question come from one "rule" frequently used in Econometrics books. Then, briefly, if some included regressors and error term are correlated we have endogeneity problem. Unfortunately some presentation do not underscore effectively what kind of "error term" are involved in this "rule".

We can intend it as the "true error term", the error term of the true model. The exogeneity assumption for OLS come from here.

Alternatively we can intend this "error term" as the error term of the misspecified model, where the misspecification can appear clearly only if the true model is known.

In real world this error term is an unobservable quantity. What you observe are the "residuals", related but different things. From residuals only we cannot discover endogeneity, in fact in OLS framework exogeneity is an untestable assumption.

EDIT: Just a warning. The problem of endogeneity (then exogeneity) is of tremendous importance in econometrics and can be write down in various version. Even for this reason the debate, and sometimes confusion, about those concepts is common. In my view concepts like endogeneity (then exogeneity) must be always related to causality and, therefore, structural concepts. I wrote something about that in this site, see here for instance:

endogenous regressor and correlation

Regression and causality in econometrics

Endogeneity in forecasting

Keeping aside the above aspects. Here I limit myself to suppose what sabiste had in his mind when wrote his question. In econometrics presentations is common to take back various problem like: omitted variables, simultaneity, measurement errors; to endogeneity problem. Shortly, endogeneity imply biasedness in some parameters.

In the "rule" the correlation between errors and included regressors are indicated as the core of the problem; the trace of him. We can read Wikipedia also:

If the independent variable is correlated with the error term in a regression model then the estimate of the regression coefficient in an ordinary least squares (OLS) regression is biased; however if the correlation is not contemporaneous, then the coefficient estimate may still be consistent.

https://en.wikipedia.org/wiki/Endogeneity_(econometrics)

at least at general level, no other conditions are added. I suppose that sabiste conflated the role of residuals with that of error terms intended as clarified above. Common mistake among neophyte.

$\endgroup$
7
  • $\begingroup$ This is a much better way of putting it. However, in addition to the correlation, you need the excluded regressor to also have a non-zero effect on the outcome to get endogeneity bias. Without that second assumption being necessary, we wouldn't even be able to run experiments, since our coin-flip based assignments is certainly correlated with something out in the world that we left out of the model. $\endgroup$
    – dimitriy
    Jul 21 '20 at 2:40
  • $\begingroup$ I do not sure about the necessity of the condition that you add. Read the EDIT on my answer. $\endgroup$
    – markowitz
    Jul 21 '20 at 15:24
  • $\begingroup$ If the condition I mentioned does not hold, $\delta=0$, so the second term would drop out in the last equation of that section. $\endgroup$
    – dimitriy
    Jul 21 '20 at 19:58
  • $\begingroup$ I know, and as I suspected you have in mind OVB story. Now, OVB is one typical source of endogeneity in the model. However the 'correlation rule' mentioned above, per se, its enough. Infact omitted variables, and other problems also, produre precisely the correlation in argument. $\endgroup$
    – markowitz
    Jul 21 '20 at 22:32
  • $\begingroup$ First, other types of endogeneity can usually be expressed as an omitted variables problem. Second, correlation is not enough for either bias or endogeneity. Take the DGP $y=1 + 2 \cdot x + 0 \cdot z + \varepsilon$. You are saying that as long as $\rho(x,z) \ne 0$ , you will have bias in the coefficient of $x$ when you leave $z$ out of the model and that $x$ will be endogenous. But that is not sufficient for bias since even though $E[x'z \vert x] !=0 $, when you multiply that by $\delta=0$, the bias term goes to zero in the plim. $\endgroup$
    – dimitriy
    Jul 22 '20 at 1:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.