How to show that $E\left[ y E(y|x) \right] = E\left[ E(y|x)E(y|x) \right]$ - linearity of conditional expectations?

Question

I'm trying to finish a proof for a review exercise and I'm asked to show that

$E\left[(y-E(y|x))(E(y|x)-f(x))\right]=0$

where $y$ is the dependent variable and $f(x)$ is a linear predictor of $y$.

I'm almost finished, but I just want to check whether or not

$E\left[ E \left[ y E(y|x)\,|\,x \right] \right]=E\left[ E(y|x)E \left[ E(y|x)\,|\,x \right] \right]$

Basically, can I take $y$ out of the conditional expectation as $E(y|x)$, or worded differently, are conditional expectations multiplacative in this way?

Ordinarily you can back a function out of the expectation if it is a function of the variable being conditioned on - i.e., $E\left[ f(x)y|x \right]=f(x)E(y|x)$, but I think what I'm trying to do above is different.

Also, a second, related question – if $E\left[E(y|x)\right]=E(y)$ by the Law of Total Expectations, then does $E\left[E(y|x)E(y|x)\right]=E(y^2)$ by the same token?

Answer 1

You answer your own question. There is nothing different than backing out function of $x$ from the conditional expectation conditioned on $x$.

Introduce definition

$$g(x)=E(y|x)$$

Then

$$Eyg(x)=E[E[yg(x)|x]]=E[g(x)E[y|x]]=E[(E[y|x])^2]$$

And we get your result. However your last statement is false. $E(y|X)$ is a random variable which is different from $y$, so its second moment should not be in general the same as $y$. To see that condition let $x$ be the random variable $1_A(w)$, where set $A\subset \Omega$ and $1_A$ is the set indicator function. Then

$$E(y|x)=\frac{Ey1_A}{P(A)}1_A+\frac{Ey1_{A^c}}{1-P(A)}1_{A^c}$$

where $A^c=\Omega\backslash A$. Then it is clear that $EE(y|x)=Ey$. However

$$[E(y|x)]^2=\left(\frac{Ey1_A}{P(A)}\right)^21_A+\left(\frac{Ey1_{A^c}}{1-P(A)}\right)^21_{A^c}$$

and taking the expectation we see that it is not equal to $Ey^2$. In fact $Ey^2$ can even be undefined, yet $E(E(y|x))^2$ exists.

Answer 2

By definition, $\newcommand{\E}{\mathrm{E}}\E[Y\mid X]$ is a measurable function of $X$ a.s. But then, $\E[Y\mid X]-f(X)$ is also a measurable function, say $g$, of $X$ a.s. Using the properties of the conditional expectation, you get what you need. $$ \E[(Y-\E[Y\mid X])(\E[Y\mid X]-f(X))] = \E[(Y-\E[Y\mid X])g(X)] $$ $$ = \E[g(X)Y] - \E[g(X)\E[Y\mid X]] \qquad \textrm{(linearity of the expectation)} $$ $$ = \E[g(X)Y] - \E[\E[g(X)Y\mid X]] \qquad (\textrm{$g(X)$ is $\sigma(X)$-measurable)} $$ $$ = \E[g(X)Y] - \E[g(X)Y] = 0 \, . \qquad \textrm{(tower property)} $$