43
$\begingroup$

I understand that the basic definition of endogeneity is that $$ X'\epsilon=0 $$ is not satisfied, but what does this mean in a real world sense? I read the Wikipedia article, with the supply and demand example, trying to make sense of it, but it didn't really help. I've heard the other description of endogenous and exogenous as being within the system and being outside the system and that still doesn't make sense to me.

$\endgroup$
  • 1
    $\begingroup$ All three of the answers below are very good (+1 to each). If you want another source of information, I discuss this topic here: Estimating $b_1x_1+b_2x_2$ instead of $b_1x_1+b_2x_2+b_3x_3$, & illustrate it w/ a simulation in R. $\endgroup$ – gung Sep 14 '13 at 23:31
  • 1
    $\begingroup$ When you have endogeneity your regression no longer has usable estimators or test statistics. $\endgroup$ – Ivan Aug 31 '16 at 19:07
  • 1
    $\begingroup$ I agree with @gung, and would like to emphasise that a complete answer would address "Usable for what purpose"? Many of the above answers deal with this question very well. $\endgroup$ – Matthew Drury Aug 31 '16 at 19:41
  • $\begingroup$ @Matthew It seems to me this post does attempt to respond to the question "what does this mean in a real world sense?" It would be nice to see the explanation fleshed out so that people could appreciate it better. $\endgroup$ – whuber Aug 31 '16 at 20:30
  • $\begingroup$ @whuber I don't know, it's so short I can't really tell. But I was thinking, for example, that the estimated model can be useful for prediction (or just association) even if you have endogeneity, so "no longer has usable estimators" seems false without clarification. $\endgroup$ – Matthew Drury Aug 31 '16 at 20:33
68
$\begingroup$

JohnRos's answer is very good. In plain English, endogeneity means you got the causation wrong. That the model you wrote down and estimated does not properly capture the way causation works in the real world. When you write:

\begin{equation} Y_i=\beta_0+\beta_1X_i+\epsilon_i \end{equation}

you can think of this equation in a number of ways. You could think of it as a convenient way of predicting $Y$ based on $X$'s values. You could think of it as a convenient way of modeling $E\{Y|X\}$. In either of these cases, there is no such thing as endogeneity, and you don't need to worry about it.

However, you can also think of the equation as embodying causation. You can think of $\beta_1$ as the answer to the question: "What would happen to $Y$ if I reached in to this system and experimentally increased $X$ by 1?" If you want to think about it that way, using OLS to estimate it amounts to assuming that:

  1. $X$ causes $Y$
  2. $\epsilon$ causes $Y$
  3. $\epsilon$ does not cause $X$
  4. $Y$ does not cause $X$
  5. Nothing which causes $\epsilon$ also causes $X$

Failure of any one of 3-5 will generally result in $E\{\epsilon|X\}\ne0$, or, not quite equivalently, ${\rm Cov}(X,\epsilon)\ne0$. Instrumental variables is a way of correcting for the fact that you got the causation wrong (by making another, different, causal assumption). A perfectly conducted randomized controlled trial is a way of forcing 3-5 to be true. If you pick $X$ randomly, then it sure ain't caused by $Y$, $\epsilon$, or anything else. So-called "natural experiment" methods are attempts to find special circumstances out in the world where 3-5 are true even when we don't think 3-5 are usually true.

In JohnRos's example, to calculate the wage value of education, you need a causal interpretation of $\beta_1$, but there are good reasons to believe that 3 or 5 is false.

Your confusion is understandable, though. It is very typical in courses on the linear model for the instructor to use the causal interpretation of $\beta_1$ I gave above while pretending not to be introducing causation, pretending that "it's all just statistics." It's a cowardly lie, but it's also very common.

In fact, it is part of a larger phenomenon in biomedicine and the social sciences. It is almost always the case that we are trying to determine the causal effect of $X$ on $Y$---that's what science is about after all. On the other hand, it is also almost always the case that there is some story you can tell leading to a conclusion that one of 3-5 is false. So, there is a kind of practiced, fluid, equivocating dishonesty in which we swat away objections by saying that we're just doing associational work and then sneak the causal interpretation back elsewhere (normally in the introduction and conclusion sections of the paper).

If you are really interested, the guy to read is Judea Perl. James Heckman is also good.

$\endgroup$
  • 5
    $\begingroup$ +1 Great explanation and commentary. Welcome to our site!. $\endgroup$ – whuber Jun 3 '13 at 14:13
  • 2
    $\begingroup$ Could you state which work by Heckman you'd recommend, for gaining a basic and solid understanding on this issue? $\endgroup$ – Kenny LJ Mar 15 '15 at 14:55
  • $\begingroup$ I have a question: how to check whether $E[\epsilon|X]=0$ or $E[\epsilon X]=0$ is true “using your data on hand (rather than your domain knowledge) which may not come from experiment, i.e., observational data set”? I feel that there is no way to test $E[\epsilon|X]=0$ or $E[\epsilon X]=0$ just use data, since $\epsilon$ is not observable, then is that true that endogeneity cannot be tested using data? $\endgroup$ – KevinKim Sep 30 '16 at 1:53
  • 1
    $\begingroup$ @KevinKim Yes. $E\{\epsilon|X\}=0$ is not testable using statistics. $\epsilon$ cannot be recovered/estimated except by doing an estimation and then making residuals. The recovery can only be done after an estimation. The recovery is only correct if the estimation is done correctly. The estimation is only done correctly if $E\{\epsilon|X\}=0$. So, circular. The information that $E\{\epsilon|X\}=0$ must come from substantive, non-statistical knowledge. An example of this is that $Cov\{X,e\}=0$ where $e$ are the OLS residuals. This is true regardless of whether $E\{\epsilon|X\}=0$. $\endgroup$ – Bill Dec 13 '16 at 13:16
  • 2
    $\begingroup$ @KevinKim That's right. And it's not just the linear model. It's all of statistics. Notice, when someone says "correlation isn't causation" they never, ever go on to tell you what is causation. Causation is theory and can only be theory. Even a (perfectly---and thus never---conducted) RCT doesn't tell you causation without theory. $\endgroup$ – Bill Dec 13 '16 at 13:49
18
$\begingroup$

Let me use an example:

Say you want to quantify the (causal) effect of education on income. You take education years and income data and regress one against the other. Did you recover what you wanted? Probably not! This is because the income is also caused by things other than education, but which are correlated to education. Let's call them "skill": We can safely assume that education years are affected by "skill", as the more skilled you are, the easier it is to gain education. So, if you regress education years on income, the estimator for the education effect absorbs the effect of "skill" and you get an overly optimistic estimate of return to education. This is to say, education's effect on income is (upward) biased because education is not exogenous to income.

Endogeneity is only a problem if you want to recover causal effects (unlike mere correlations). Also- if you can design an experiment, you can guarantee that ${\rm Cov}(X,\epsilon)=0$ by random assignment. Sadly, this is typically impossible in social sciences.

$\endgroup$
  • 1
    $\begingroup$ Thanks for the example and the explanation. I am still a bit clueless about what endogeneity and exogeneity mean in plain English. What exactly do i mean when I say that a variable is endogenous or for that matter exogenous. $\endgroup$ – user25901 May 21 '13 at 10:44
  • $\begingroup$ @ JohnRos You wrote "Endogeneity is only a problem if you want to recover causal effects" then it seems me that is also possible to say that: "exogeneity imply causality" ... I never read that phrase ... however Its right ? If it is correct it seems me that many textbook, sometimes implicitly, suppose causal inference as normal goals. $\endgroup$ – markowitz Jul 6 '17 at 14:16
  • $\begingroup$ @markowitz: Whenever you are inferring on regression coefficients, it is implied you want causality. If you only want predictions, the value of the coefficients does not really matter, provided that predictions are good. It is true that classical textbooks do not make this distinction because before the task of prediction is not "basic science" but rather more "engineering" (and forgive me for this crude generalization) $\endgroup$ – JohnRos Jul 7 '17 at 7:45
  • $\begingroup$ Thanks JohnRos, let me ask another question about a related point. The problem of biased estimation of the coefficients make sense only in causation regression model while for prediction goals definitely are not. Its right? I ask this because this point is not clear in any place. $\endgroup$ – markowitz Jul 7 '17 at 17:42
8
$\begingroup$

User25901 is looking for a straight-forward simple, real-world explanation what the terms exogenous and endogenous mean. Responding with arcane examples or mathematical definitions does not really answer the question that was asked.

How do I get a gut understanding of these two terms?

Here's what I came up with:

Exo - external, outside Endo - internal, inside -genous - originating in

Exogeneous: A variable is exogenous to a model if it is not determined by other parameters and variables in the model, but is set externally and any changes to it come from external forces.

Endogenous: A variable is endogenous in a model if it is at least partly function of other parameters and variables in a model.

$\endgroup$
  • 7
    $\begingroup$ These are reasonable intuitive definitions, but there is no need to be so dismissive of the other answers. $\endgroup$ – gung Apr 21 '14 at 19:36
  • 3
    $\begingroup$ Appealing to etymology can give one useful handles for remembering what technical terms mean (it works well for me), but using etymology to justify them is to be avoided. Quite a few terms (in statistics and elsewhere) are properly understood only through careful study of their mathematical definitions. Understanding this answer requires a clear conception of the intended uses of words and phrases like "determined by," "set externally," "changes to," "external forces," and "partly [a] function," none of which are immediately apparent or unambiguous. $\endgroup$ – whuber Apr 21 '14 at 21:38
6
$\begingroup$

The OLS regression, by construction, gives $X'\epsilon=0$. Actually that is not correct. It gives $X'\hat\epsilon=0$ by construction. Your estimated residuals are uncorrelated with your regressors, but your estimated residuals are "wrong" in a sense.

If the true data-generating-process operates by $Y=\alpha +\beta X + \gamma Z + {\rm noise}$, and $Z$ is correlated with $X$, then $X'{\rm noise} \neq 0$ if you fit a regression leaving out $Z$. Of course, the estimated residuals will be uncorrelated with $X$. They always are, the same way that $\log(e^x)=x$. It is just a mathematical fact. This is the omitted variable bias.

Say that $I$ is randomly assigned. Maybe it is the day of week that people are born. Maybe it is an actual experiment. It is anything uncorrelated with $Y$ that predicts $X$. You can then use the randomness of $I$ to predict $X$, and then use that predicted $X$ to fit a model to $Y$.

That is two stage least squares, which is almost the same as IV.

$\endgroup$
  • $\begingroup$ As I understand isn't 2SLS one way to do IV, apologies if I'm mistaken. $\endgroup$ – user25901 May 21 '13 at 10:42
  • $\begingroup$ 2SLS standard errors are wrong. I forget why or how, but you'd probably find something if you google "IV 2SLS standard errors". Most software packages implement 2sls with the solve(t(z)%*%(x)%*%t(z)%*%y method $\endgroup$ – generic_user May 21 '13 at 11:13
  • 1
    $\begingroup$ 2SLS standard errors are wrong because the input to the final stage (say $\hat{X}$) doesn't reflect the actual variance of $X$. Corrected SEs adjust for this. $\endgroup$ – MichaelChirico Feb 17 '17 at 16:59
  • $\begingroup$ Thanks. I was fresh out of applied econometrics when I wrote this. $\endgroup$ – generic_user Feb 17 '17 at 18:02
-1
$\begingroup$

In regression we want to capture the quantitative impact of an independent variable (which we assume is exogenous and not being itself dependent on something else) on an identified dependent variable. We want to know what net effect an exogenous variable has on a dependent variable- meaning the independent variable should be free of any influence from another variable. A quick way to see if the regression is suffering from the problem of endogeneity is to check the correlation between the independent variable and the residuals. But this is just a rough check otherwise formal tests of endogeneity need to be undertaken.

$\endgroup$
  • 3
    $\begingroup$ This isn't true. The correlation between the residuals and the explanatory variables from a regression is zero by construction. This is not a test for endogeneity. $\endgroup$ – Andy May 1 '15 at 18:42
  • $\begingroup$ @Andy I agree with you. Then my question is: is there a way to test endogeneity $E[\epsilon X]=0$ just using data? where $\epsilon$ is not the residual but from $y=b_0+b_1x+\epsilon$, i.e., the model that you believed that generates the data, so $\epsilon$ is not observable. In addition, I feel that Amon wants to say that you can empirically check whether $E[\hat{e}_i|x]=0$, where $\hat{e}_i$ is the residual. If $E[\hat{e}_i|x]=0$ is roughly true, then you can claim $\hat{b}_0+\hat{b}_1x$ probably capture the conditional mean and hence, there is not much endogenity issue, am I correct? $\endgroup$ – KevinKim Sep 30 '16 at 2:01

protected by kjetil b halvorsen Nov 12 '17 at 15:00

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.