I understand that the basic definition of endogeneity is that $$ X'\epsilon=0 $$ is not satisfied, but what does this mean in a real world sense? I read the Wikipedia article, with the supply and demand example, trying to make sense of it, but it didn't really help. I've heard the other description of endogenous and exogenous as being within the system and being outside the system and that still doesn't make sense to me.
$\begingroup$
$\endgroup$
8
5 Answers
$\begingroup$
$\endgroup$
6
JohnRos's answer is very good. In plain English, endogeneity means you got the causation wrong. That the model you wrote down and estimated does not properly capture the way causation works in the real world. When you write:
\begin{equation} Y_i=\beta_0+\beta_1X_i+\epsilon_i \end{equation}
you can think of this equation in a number of ways. You could think of it as a convenient way of predicting $Y$ based on $X$'s values. You could think of it as a convenient way of modeling $E\{Y|X\}$. In either of these cases, there is no such thing as endogeneity, and you don't need to worry about it.
However, you can also think of the equation as embodying causation. You can think of $\beta_1$ as the answer to the question: "What would happen to $Y$ if I reached in to this system and experimentally increased $X$ by 1?" If you want to think about it that way, using OLS to estimate it amounts to assuming that:
- $X$ causes $Y$
- $\epsilon$ causes $Y$
- $\epsilon$ does not cause $X$
- $Y$ does not cause $X$
- Nothing which causes $\epsilon$ also causes $X$
Failure of any one of 3-5 will generally result in $E\{\epsilon|X\}\ne0$, or, not quite equivalently, ${\rm Cov}(X,\epsilon)\ne0$. Instrumental variables is a way of correcting for the fact that you got the causation wrong (by making another, different, causal assumption). A perfectly conducted randomized controlled trial is a way of forcing 3-5 to be true. If you pick $X$ randomly, then it sure ain't caused by $Y$, $\epsilon$, or anything else. So-called "natural experiment" methods are attempts to find special circumstances out in the world where 3-5 are true even when we don't think 3-5 are usually true.
In JohnRos's example, to calculate the wage value of education, you need a causal interpretation of $\beta_1$, but there are good reasons to believe that 3 or 5 is false.
Your confusion is understandable, though. It is very typical in courses on the linear model for the instructor to use the causal interpretation of $\beta_1$ I gave above while pretending not to be introducing causation, pretending that "it's all just statistics." It's a cowardly lie, but it's also very common.
In fact, it is part of a larger phenomenon in biomedicine and the social sciences. It is almost always the case that we are trying to determine the causal effect of $X$ on $Y$---that's what science is about after all. On the other hand, it is also almost always the case that there is some story you can tell leading to a conclusion that one of 3-5 is false. So, there is a kind of practiced, fluid, equivocating dishonesty in which we swat away objections by saying that we're just doing associational work and then sneak the causal interpretation back elsewhere (normally in the introduction and conclusion sections of the paper).
If you are really interested, the guy to read is Judea Perl. James Heckman is also good.
-
6$\begingroup$ +1 Great explanation and commentary. Welcome to our site!. $\endgroup$– whuber ♦Commented Jun 3, 2013 at 14:13
-
2$\begingroup$ Could you state which work by Heckman you'd recommend, for gaining a basic and solid understanding on this issue? $\endgroup$– user46481Commented Mar 15, 2015 at 14:55
-
$\begingroup$ I have a question: how to check whether $E[\epsilon|X]=0$ or $E[\epsilon X]=0$ is true “using your data on hand (rather than your domain knowledge) which may not come from experiment, i.e., observational data set”? I feel that there is no way to test $E[\epsilon|X]=0$ or $E[\epsilon X]=0$ just use data, since $\epsilon$ is not observable, then is that true that endogeneity cannot be tested using data? $\endgroup$– KevinKimCommented Sep 30, 2016 at 1:53
-
2$\begingroup$ @KevinKim Yes. $E\{\epsilon|X\}=0$ is not testable using statistics. $\epsilon$ cannot be recovered/estimated except by doing an estimation and then making residuals. The recovery can only be done after an estimation. The recovery is only correct if the estimation is done correctly. The estimation is only done correctly if $E\{\epsilon|X\}=0$. So, circular. The information that $E\{\epsilon|X\}=0$ must come from substantive, non-statistical knowledge. An example of this is that $Cov\{X,e\}=0$ where $e$ are the OLS residuals. This is true regardless of whether $E\{\epsilon|X\}=0$. $\endgroup$– BillCommented Dec 13, 2016 at 13:16
-
3$\begingroup$ @KevinKim That's right. And it's not just the linear model. It's all of statistics. Notice, when someone says "correlation isn't causation" they never, ever go on to tell you what is causation. Causation is theory and can only be theory. Even a (perfectly---and thus never---conducted) RCT doesn't tell you causation without theory. $\endgroup$– BillCommented Dec 13, 2016 at 13:49
$\begingroup$
$\endgroup$
6
Let me use an example:
Say you want to quantify the (causal) effect of education on income. You take education years and income data and regress one against the other. Did you recover what you wanted? Probably not! This is because the income is also caused by things other than education, but which are correlated to education. Let's call them "skill": We can safely assume that education years are affected by "skill", as the more skilled you are, the easier it is to gain education. So, if you regress education years on income, the estimator for the education effect absorbs the effect of "skill" and you get an overly optimistic estimate of return to education. This is to say, education's effect on income is (upward) biased because education is not exogenous to income.
Endogeneity is only a problem if you want to recover causal effects (unlike mere correlations). Also- if you can design an experiment, you can guarantee that ${\rm Cov}(X,\epsilon)=0$ by random assignment. Sadly, this is typically impossible in social sciences.
-
1$\begingroup$ Thanks for the example and the explanation. I am still a bit clueless about what endogeneity and exogeneity mean in plain English. What exactly do i mean when I say that a variable is endogenous or for that matter exogenous. $\endgroup$ Commented May 21, 2013 at 10:44
-
$\begingroup$ @ JohnRos You wrote "Endogeneity is only a problem if you want to recover causal effects" then it seems me that is also possible to say that: "exogeneity imply causality" ... I never read that phrase ... however Its right ? If it is correct it seems me that many textbook, sometimes implicitly, suppose causal inference as normal goals. $\endgroup$ Commented Jul 6, 2017 at 14:16
-
$\begingroup$ @markowitz: Whenever you are inferring on regression coefficients, it is implied you want causality. If you only want predictions, the value of the coefficients does not really matter, provided that predictions are good. It is true that classical textbooks do not make this distinction because before the task of prediction is not "basic science" but rather more "engineering" (and forgive me for this crude generalization) $\endgroup$– JohnRosCommented Jul 7, 2017 at 7:45
-
$\begingroup$ Thanks JohnRos, let me ask another question about a related point. The problem of biased estimation of the coefficients make sense only in causation regression model while for prediction goals definitely are not. Its right? I ask this because this point is not clear in any place. $\endgroup$ Commented Jul 7, 2017 at 17:42
-
$\begingroup$ Wouldn't endogeneity be the reflexive-adjective, of the right hand side functional or absolute operation, over a hypothesis? For instance, in a case or class resolution: I would like to write, "if you have order, you might not have endogeneocity." I would call that a solution, not a problem, along any verb. Endogeneity is inactionable, as a problem. But as a solution, the etymological use of the word is clear and plausibly bona fide. $\endgroup$ Commented Jan 22, 2022 at 23:18
$\begingroup$
$\endgroup$
2
User25901 is looking for a straight-forward, simple, real-world explanation of what the terms exogenous and endogenous mean. To respond with arcane examples or mathematical definitions is to not really answer the question that was asked.
How do I, 'get a gut understanding of these two terms?'
Here's what I came up with:
Exo - external, outside Endo - internal, inside -genous - originating in
Exogenous: A variable is exogenous to a model if it is not determined by other parameters and variables in the model, but is set externally and any changes to it come from external forces.
Endogenous: A variable is endogenous in a model if it is at least partly function of other parameters and variables in a model.
Therefore, exo-geneity and exo-geneocity is the reflexive adjective of the right-hand-side cause, describing its effect on others, rather than its own integrality or derivations.
-
10$\begingroup$ These are reasonable intuitive definitions, but there is no need to be so dismissive of the other answers. $\endgroup$ Commented Apr 21, 2014 at 19:36
-
7$\begingroup$ Appealing to etymology can give one useful handles for remembering what technical terms mean (it works well for me), but using etymology to justify them is to be avoided. Quite a few terms (in statistics and elsewhere) are properly understood only through careful study of their mathematical definitions. Understanding this answer requires a clear conception of the intended uses of words and phrases like "determined by," "set externally," "changes to," "external forces," and "partly [a] function," none of which are immediately apparent or unambiguous. $\endgroup$– whuber ♦Commented Apr 21, 2014 at 21:38
$\begingroup$
$\endgroup$
4
The OLS regression, by construction, gives $X'\epsilon=0$. Actually that is not correct. It gives $X'\hat\epsilon=0$ by construction. Your estimated residuals are uncorrelated with your regressors, but your estimated residuals are "wrong" in a sense.
If the true data-generating-process operates by $Y=\alpha +\beta X + \gamma Z + {\rm noise}$, and $Z$ is correlated with $X$, then $X'{\rm noise} \neq 0$ if you fit a regression leaving out $Z$. Of course, the estimated residuals will be uncorrelated with $X$. They always are, the same way that $\log(e^x)=x$. It is just a mathematical fact. This is the omitted variable bias.
Say that $I$ is randomly assigned. Maybe it is the day of week that people are born. Maybe it is an actual experiment. It is anything uncorrelated with $Y$ that predicts $X$. You can then use the randomness of $I$ to predict $X$, and then use that predicted $X$ to fit a model to $Y$.
That is two stage least squares, which is almost the same as IV.
gung - Reinstate Monica
148k89 gold badges406 silver badges717 bronze badges
answered May 21, 2013 at 10:27
-
$\begingroup$ As I understand isn't 2SLS one way to do IV, apologies if I'm mistaken. $\endgroup$ Commented May 21, 2013 at 10:42
-
$\begingroup$ 2SLS standard errors are wrong. I forget why or how, but you'd probably find something if you google "IV 2SLS standard errors". Most software packages implement 2sls with the solve(t(z)%*%(x)%*%t(z)%*%y method $\endgroup$ Commented May 21, 2013 at 11:13
-
1$\begingroup$ 2SLS standard errors are wrong because the input to the final stage (say $\hat{X}$) doesn't reflect the actual variance of $X$. Corrected SEs adjust for this. $\endgroup$ Commented Feb 17, 2017 at 16:59
-
$\begingroup$ Thanks. I was fresh out of applied econometrics when I wrote this. $\endgroup$ Commented Feb 17, 2017 at 18:02
$\begingroup$
$\endgroup$
Think of a system as $x,y$. When we're trying to explain it by a model $y=f(x)+\varepsilon$, is the error $\varepsilon$ a part of the system or not?
When the error is not part of the system, we call it exogenous, i.e. it's added to $f(x)$ after $x$ had its input into the system.
When the error is a part of the system, we call it endogenous, i.e. not only it enters $y$ after $f(x)$, it also enters $x$ itself somehow before $f(.)$ is applied to it.
This makes $endogenous$ models troublesome, for they interfere with our attempts to estimate the function $f(.)$.
R. $\endgroup$