The question

I wanted to prove the equation, but I'm somehow stuck in the middle of the process.

To solve the right side, I have

$[{E[(Y|X)^2}] + E[Y^2] - 2E[Y*E[Y|X]] + E[{E[Y|X]}^2]$

==> $2 E[{E[(Y|X)^2}] + E[Y^2] - 2E[Y*E[Y|X]]$

==> $2 E[{E[Y^2|X] - V[Y|X]}]$ (theorem: $V(Y|X) = E[Y^2|X] - E[Y|X]^2$ ) $+ E[Y^2] - 2E[Y*E[Y|X]]$

==> $2 ( E[Y^2] (E[E[Y^2|X]] = E[Y^2]) - E[V[Y|X] ) + E[Y^2] - 2 E[Y*E[Y|X]]$

==> $3E[Y^2] - 2 E[V[Y|X] - 2 E[Y*E[Y|X]]$ \end{align} but I dont know how to prove the equation above to be $E[Y^2]$. I assume we use the Law of Total Variance: $V(Y) = E[V[Y|X] + V[E[Y|X]$. But dont't know how.

Thanks for the help.

up vote 3 down vote accepted

I guess you already have answers, but $$ E(Y^2) = E[E(Y^2 \mid X)] = E[\operatorname{Var}(Y \mid X) + E(Y \mid X)^2] = E\left\{(Y - E[Y \mid X])^2\right\} + E\left\{E(Y \mid X)^2\right\} $$ where we have just applied the fact that $E(Z^2) = \operatorname{Var}(Z) + [E(Z)]^2$ to evaluate $E(Y^2 \mid X)$; this is valid, as long as one believes that $E(\cdot \mid X)$ behaves like an expectation operator should. This approach doesn't require any kind of insight about the fact that $Y - E(Y \mid X)$ is orthogonal to $E(Y \mid X)$.

If we define $\|Z\| = \sqrt{E(Z^2)}$ as the length of $Z$ then this of course says that $$ \|Y - E(Y \mid X)\|^2 + \|E(Y \mid X)\|^2 = \|Y\|^2, $$ which is the Pythagorean theorem (the squared length of $Y$ is equal to the sum of the squared lengths of $Y - E(Y \mid X)$ and $E(Y \mid X)$.

What a marvelous problem. Your instructor (with his towering intellect) has given you quite a gem to work on.

If you add and subtract $E[Y\mid X]$ to $Y$, then you can write $$ \{Y- E[Y \mid X]\} + E[Y|X] = Y \tag{1}. $$ To see the connection with the Pythagorean Theorem, think of this equation as $$ A + B = C, $$ where $A = \{Y- E[Y \mid X]\}$, $B =E[Y|X]$ and $C = Y$. This notation might be familiar if you were thinking of $A$ , $B$ and $C$ as vectors, where $A$ is the flat vector on the bottom of the right triangle, $B$ is the vertical piece, and $C$ is the hypotenuse.

The Pythagorean theorem for vectors says, that as long as $A$ and $B$ are orthogonal, we have the length of $A$ squared plus the length of $B$ squared equals the length of $C$ squared. How can random variables be orthogonal, and how do we take the length of them?

@KDG has the right idea: the squared length of a random vector is the average of its square, so square both sides (1), and then take the average on both sides, and you will arrive at the result. The orthogonal idea has to deal with why $$ E\left[\{Y- E[Y \mid X]\} E[Y|X] \right] = E\left[A B \right] = 0. $$ This would solve the problem by making that cross term have mean zero. Also, it would help you solve it from the second line of your work because it shows $2 E[{E[(Y|X)^2}] - 2E[Y*E[Y|X]] = 0$, and so you're left with $E[Y^2]$.

Perhaps your (terrific, magnificent, amazing, etc.) instructor worked this out in class by showing the covariance between these two pieces is $0$. Perhaps it's on slide 14 of the notes from section 5.3. Also, I'm sure if you asked him, he would tell you he's very glad to see his students working very hard on their homework.

  • Hi sir, I like the explanation you give. However, I don't remember we covered Pythagorean Theorem of means of random variables in our class. So I don't know how to derive or apply it to the homework. I solved the problem using guy's solution, which involves less theorems. But I would like to learn more ways of doing this problem. Can you elaborate a little on how equation (1) is applied to the question in particular? Thank you very much for your help. – Muchdecal Sep 15 at 2:04

$(Y-E(Y|X)+E(Y|X))^2=(Y-E(Y|X))^2+E(Y|X)^2+2(Y-E(Y|X))E(Y|X)$

Take expectation on both sides and we can show that $E(Y)=E(E(Y|X))$.

Your Answer

 
discard

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.