0
$\begingroup$

I'm trying to read mostly harmless econometrics. They give a proof for the Anova Theorem in which they use that fact that E[y|X] and $\epsilon$ are uncorrelated. I assume they say that as before they have shown that E[h(X)$\epsilon$] is 0. So I guess the conditional expectation is just one possible function of X? This confuses me, however. After all $\epsilon$ is defined as y - E[y|X], how can they be uncorrelated?

$\endgroup$
  • $\begingroup$ Actually in regression Y=aX+b+ $\epsilon$ where $\epsilon$ is assumed to be uncorrelated with X with mean 0. Then E(Y|X=x)=ax+b and not $\epsilon$. So I can't make sense of your question. $\endgroup$ – Michael Chernick Oct 22 '17 at 20:09
0
$\begingroup$

Per the above comment, I think you asked the question in a way that your notation is wrong. It sounds like you are asking, in a model that is $Y=c+aX+\epsilon$ why $\epsilon$ is independent/uncorrelated with the other variables, namely $X$ and $E[Y|X]$ - as well as why each $\epsilon_i$ is independent of the others.

It is an assumption. If you assume that, then normal regression (OLS, or Ordinary Least Sqaures) (1) works in a straighforward way and (2) has desirable/optimal properties (like giving unbiased estimators).

If that weren't true, you would need to fiddle with the data or do more complex regressions. For example, if the $\epsilon$'s were correlated with each other, you have a problem called autocorrelation.

In theory one ought to review the $\epsilon$'s to make sure they are independent and normally distributed. When I asked an econometrics professor once why people don't do that, he laughed and said "because they wouldn't be what you want, and you couldn't publish".

$\endgroup$
  • $\begingroup$ this is exactly what i thought. but the authors made it seems like you could prove (and not assume) that it was uncorrelated. I found the pages here (econ.lse.ac.uk/staff/spischke/mhe/ex_ch3.pdf). I thought the key lies in Theorem 3.1.1 proof ii. After all, if epsilon is uncorrelated with ANY function of X then it also has to be uncorrelated to the traditional linear regression you wrote? I'm probably misunderstanding something. $\endgroup$ – leo Oct 22 '17 at 20:50
  • $\begingroup$ I don't love the pedagogical approach they are taking, and I suspect this is causing confusion. What they are really saying is that, for every $i$ (corresponding to some number of years of schooling), the random distribution for that year has an average (which is all $E[Y_i|X_i]$ is, really - the average for data for index $i$), and then there is some random distribution $\epsilon_i$ about that mean. That is certainly sensible. But the issues that come up in regression often occur when the $\epsilon_i$'s are not iid normally distributed, or correlated with each other. $\endgroup$ – eSurfsnake Oct 23 '17 at 1:21
  • $\begingroup$ Hey eSurfsnake, I have follow up question which i posted here: stats.stackexchange.com/questions/311791/…, you have been so helpful with this one would you mind taking a look at the other? :) $\endgroup$ – leo Nov 4 '17 at 17:35
0
$\begingroup$

In the conditional expectation function set up of a relation,

$$y = E(y \mid x) + e$$

it is true that

$$E(e \mid x) = 0 \implies E[h(x)e] = 0$$

because

$$E[h(x)e] = E[E(h(x)e\mid x)] = E[h(x)E(e\mid x)] = E[h(x)\cdot 0] = 0$$

Now let's examine

$$E[E(y\mid x) \cdot e]$$

To avoid confusion denote $E(y\mid x) \equiv Z$. Then applying the same tatic as before,

$$E[Z \cdot e] = E[ E(Ze\mid x)]= E[Z E(e\mid x)] = E[Z \cdot 0] = 0$$

And yes, this holds because $E(y \mid x)$ is just another function of $x$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.