1
$\begingroup$

I'm trying to finish a proof for a review exercise and I'm asked to show that

$E\left[(y-E(y|x))(E(y|x)-f(x))\right]=0$

where $y$ is the dependent variable and $f(x)$ is a linear predictor of $y$.

I'm almost finished, but I just want to check whether or not

$E\left[ E \left[ y E(y|x)\,|\,x \right] \right]=E\left[ E(y|x)E \left[ E(y|x)\,|\,x \right] \right]$

Basically, can I take $y$ out of the conditional expectation as $E(y|x)$, or worded differently, are conditional expectations multiplacative in this way?

Ordinarily you can back a function out of the expectation if it is a function of the variable being conditioned on - i.e., $E\left[ f(x)y|x \right]=f(x)E(y|x)$, but I think what I'm trying to do above is different.

Also, a second, related question – if $E\left[E(y|x)\right]=E(y)$ by the Law of Total Expectations, then does $E\left[E(y|x)E(y|x)\right]=E(y^2)$ by the same token?

$\endgroup$
7
$\begingroup$

You answer your own question. There is nothing different than backing out function of $x$ from the conditional expectation conditioned on $x$.

Introduce definition

$$g(x)=E(y|x)$$

Then

$$Eyg(x)=E[E[yg(x)|x]]=E[g(x)E[y|x]]=E[(E[y|x])^2]$$

And we get your result. However your last statement is false. $E(y|X)$ is a random variable which is different from $y$, so its second moment should not be in general the same as $y$. To see that condition let $x$ be the random variable $1_A(w)$, where set $A\subset \Omega$ and $1_A$ is the set indicator function. Then

$$E(y|x)=\frac{Ey1_A}{P(A)}1_A+\frac{Ey1_{A^c}}{1-P(A)}1_{A^c}$$

where $A^c=\Omega\backslash A$. Then it is clear that $EE(y|x)=Ey$. However

$$[E(y|x)]^2=\left(\frac{Ey1_A}{P(A)}\right)^21_A+\left(\frac{Ey1_{A^c}}{1-P(A)}\right)^21_{A^c}$$

and taking the expectation we see that it is not equal to $Ey^2$. In fact $Ey^2$ can even be undefined, yet $E(E(y|x))^2$ exists.

$\endgroup$
5
  • $\begingroup$ Thanks a lot for the reply. I guess I wasn't thinking of $E(y|x)$ as a "function of x" in that sense, but I see what you mean. I've fixed a typo in the second part of the question - $y$ and $y^2$ should be the unconditional means $E(y)$ and $E(y^2)$, respectively. $\endgroup$ Jul 21 '11 at 6:55
  • $\begingroup$ @jefflovejapan, we can back out function of $x$ from the conditional expectation, because sigma algebra generated by random variable $f(x)$ is a subset of sigma algebra of $x$. $\endgroup$
    – mpiktas
    Jul 21 '11 at 7:01
  • $\begingroup$ – thanks a lot, that's very helpful. I'm not familiar with the notation from the second part of your answer though - does this come from probability theory? Am I reading it right - $A^c$ is the complement of omega in $A$? I'd been putting off reading probability theory for as long as possible, but I don't know if I'll get away with it for much longer. $\endgroup$ Jul 21 '11 at 11:11
  • $\begingroup$ @jefflovejapan, yes you can say that it comes from probability theory. $A^c$ is the complement of $A$ in $\Omega$. $\endgroup$
    – mpiktas
    Jul 21 '11 at 11:23
  • $\begingroup$ @jefflovejapan, you're welcome. $\endgroup$
    – mpiktas
    Jul 21 '11 at 11:34
2
$\begingroup$

By definition, $\newcommand{\E}{\mathrm{E}}\E[Y\mid X]$ is a measurable function of $X$ a.s. But then, $\E[Y\mid X]-f(X)$ is also a measurable function, say $g$, of $X$ a.s. Using the properties of the conditional expectation, you get what you need. $$ \E[(Y-\E[Y\mid X])(\E[Y\mid X]-f(X))] = \E[(Y-\E[Y\mid X])g(X)] $$ $$ = \E[g(X)Y] - \E[g(X)\E[Y\mid X]] \qquad \textrm{(linearity of the expectation)} $$ $$ = \E[g(X)Y] - \E[\E[g(X)Y\mid X]] \qquad (\textrm{$g(X)$ is $\sigma(X)$-measurable)} $$ $$ = \E[g(X)Y] - \E[g(X)Y] = 0 \, . \qquad \textrm{(tower property)} $$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.