# How to show that $E\left[ y E(y|x) \right] = E\left[ E(y|x)E(y|x) \right]$ - linearity of conditional expectations?

I'm trying to finish a proof for a review exercise and I'm asked to show that

$E\left[(y-E(y|x))(E(y|x)-f(x))\right]=0$

where $y$ is the dependent variable and $f(x)$ is a linear predictor of $y$.

I'm almost finished, but I just want to check whether or not

$E\left[ E \left[ y E(y|x)\,|\,x \right] \right]=E\left[ E(y|x)E \left[ E(y|x)\,|\,x \right] \right]$

Basically, can I take $y$ out of the conditional expectation as $E(y|x)$, or worded differently, are conditional expectations multiplacative in this way?

Ordinarily you can back a function out of the expectation if it is a function of the variable being conditioned on - i.e., $E\left[ f(x)y|x \right]=f(x)E(y|x)$, but I think what I'm trying to do above is different.

Also, a second, related question – if $E\left[E(y|x)\right]=E(y)$ by the Law of Total Expectations, then does $E\left[E(y|x)E(y|x)\right]=E(y^2)$ by the same token?

You answer your own question. There is nothing different than backing out function of $x$ from the conditional expectation conditioned on $x$.

Introduce definition

$$g(x)=E(y|x)$$

Then

$$Eyg(x)=E[E[yg(x)|x]]=E[g(x)E[y|x]]=E[(E[y|x])^2]$$

And we get your result. However your last statement is false. $E(y|X)$ is a random variable which is different from $y$, so its second moment should not be in general the same as $y$. To see that condition let $x$ be the random variable $1_A(w)$, where set $A\subset \Omega$ and $1_A$ is the set indicator function. Then

$$E(y|x)=\frac{Ey1_A}{P(A)}1_A+\frac{Ey1_{A^c}}{1-P(A)}1_{A^c}$$

where $A^c=\Omega\backslash A$. Then it is clear that $EE(y|x)=Ey$. However

$$[E(y|x)]^2=\left(\frac{Ey1_A}{P(A)}\right)^21_A+\left(\frac{Ey1_{A^c}}{1-P(A)}\right)^21_{A^c}$$

and taking the expectation we see that it is not equal to $Ey^2$. In fact $Ey^2$ can even be undefined, yet $E(E(y|x))^2$ exists.

• Thanks a lot for the reply. I guess I wasn't thinking of $E(y|x)$ as a "function of x" in that sense, but I see what you mean. I've fixed a typo in the second part of the question - $y$ and $y^2$ should be the unconditional means $E(y)$ and $E(y^2)$, respectively. Jul 21 '11 at 6:55
• @jefflovejapan, we can back out function of $x$ from the conditional expectation, because sigma algebra generated by random variable $f(x)$ is a subset of sigma algebra of $x$. Jul 21 '11 at 7:01
• – thanks a lot, that's very helpful. I'm not familiar with the notation from the second part of your answer though - does this come from probability theory? Am I reading it right - $A^c$ is the complement of omega in $A$? I'd been putting off reading probability theory for as long as possible, but I don't know if I'll get away with it for much longer. Jul 21 '11 at 11:11
• @jefflovejapan, yes you can say that it comes from probability theory. $A^c$ is the complement of $A$ in $\Omega$. Jul 21 '11 at 11:23
• @jefflovejapan, you're welcome. Jul 21 '11 at 11:34

By definition, $\newcommand{\E}{\mathrm{E}}\E[Y\mid X]$ is a measurable function of $X$ a.s. But then, $\E[Y\mid X]-f(X)$ is also a measurable function, say $g$, of $X$ a.s. Using the properties of the conditional expectation, you get what you need. $$\E[(Y-\E[Y\mid X])(\E[Y\mid X]-f(X))] = \E[(Y-\E[Y\mid X])g(X)]$$ $$= \E[g(X)Y] - \E[g(X)\E[Y\mid X]] \qquad \textrm{(linearity of the expectation)}$$ $$= \E[g(X)Y] - \E[\E[g(X)Y\mid X]] \qquad (\textrm{g(X) is \sigma(X)-measurable)}$$ $$= \E[g(X)Y] - \E[g(X)Y] = 0 \, . \qquad \textrm{(tower property)}$$