I'm trying to read mostly harmless econometrics. They give a proof for the Anova Theorem in which they use that fact that E[y|X] and $\epsilon$ are uncorrelated. I assume they say that as before they have shown that E[h(X)$\epsilon$] is 0. So I guess the conditional expectation is just one possible function of X? This confuses me, however. After all $\epsilon$ is defined as y - E[y|X], how can they be uncorrelated?
1
-
$\begingroup$ Actually in regression Y=aX+b+ $\epsilon$ where $\epsilon$ is assumed to be uncorrelated with X with mean 0. Then E(Y|X=x)=ax+b and not $\epsilon$. So I can't make sense of your question. $\endgroup$– Michael R. ChernickCommented Oct 22, 2017 at 20:09
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
3
Per the above comment, I think you asked the question in a way that your notation is wrong. It sounds like you are asking, in a model that is $Y=c+aX+\epsilon$ why $\epsilon$ is independent/uncorrelated with the other variables, namely $X$ and $E[Y|X]$ - as well as why each $\epsilon_i$ is independent of the others.
It is an assumption. If you assume that, then normal regression (OLS, or Ordinary Least Sqaures) (1) works in a straighforward way and (2) has desirable/optimal properties (like giving unbiased estimators).
If that weren't true, you would need to fiddle with the data or do more complex regressions. For example, if the $\epsilon$'s were correlated with each other, you have a problem called autocorrelation.
In theory one ought to review the $\epsilon$'s to make sure they are independent and normally distributed. When I asked an econometrics professor once why people don't do that, he laughed and said "because they wouldn't be what you want, and you couldn't publish".
-
$\begingroup$ this is exactly what i thought. but the authors made it seems like you could prove (and not assume) that it was uncorrelated. I found the pages here (econ.lse.ac.uk/staff/spischke/mhe/ex_ch3.pdf). I thought the key lies in Theorem 3.1.1 proof ii. After all, if epsilon is uncorrelated with ANY function of X then it also has to be uncorrelated to the traditional linear regression you wrote? I'm probably misunderstanding something. $\endgroup$– leoCommented Oct 22, 2017 at 20:50
-
$\begingroup$ I don't love the pedagogical approach they are taking, and I suspect this is causing confusion. What they are really saying is that, for every $i$ (corresponding to some number of years of schooling), the random distribution for that year has an average (which is all $E[Y_i|X_i]$ is, really - the average for data for index $i$), and then there is some random distribution $\epsilon_i$ about that mean. That is certainly sensible. But the issues that come up in regression often occur when the $\epsilon_i$'s are not iid normally distributed, or correlated with each other. $\endgroup$ Commented Oct 23, 2017 at 1:21
-
$\begingroup$ Hey eSurfsnake, I have follow up question which i posted here: stats.stackexchange.com/questions/311791/…, you have been so helpful with this one would you mind taking a look at the other? :) $\endgroup$– leoCommented Nov 4, 2017 at 17:35
$\begingroup$
$\endgroup$
In the conditional expectation function set up of a relation,
$$y = E(y \mid x) + e$$
it is true that
$$E(e \mid x) = 0 \implies E[h(x)e] = 0$$
because
$$E[h(x)e] = E[E(h(x)e\mid x)] = E[h(x)E(e\mid x)] = E[h(x)\cdot 0] = 0$$
Now let's examine
$$E[E(y\mid x) \cdot e]$$
To avoid confusion denote $E(y\mid x) \equiv Z$. Then applying the same tatic as before,
$$E[Z \cdot e] = E[ E(Ze\mid x)]= E[Z E(e\mid x)] = E[Z \cdot 0] = 0$$
And yes, this holds because $E(y \mid x)$ is just another function of $x$.
answered Feb 26, 2018 at 22:21