I would like to ask for the proof of $E[E[X|Y]|Y]=E[X|Y]$

Per my understanding (for discrete case):

because $E[X|Y]=g(Y)$

hence, $E[E[X|Y]|Y] = E[g(Y)|Y]= \sum_y g(y)*p(y|y)=\sum_y g(y)=\sum E(X|Y=y)$

I could not arrive to the correct result $E[E[X|Y]|Y]=E[X|Y]$, it is very nice if someone can tell me what's wrong in my demonstration.

  • $\begingroup$ You have kinda already arrived at the result you wish to show. Note that $E[g(Y) | Y=y] = g(y)$. $\endgroup$ Apr 15 '21 at 4:53
  • $\begingroup$ I think the issue is when you say $E[g(Y)|Y] = \sum_y g(y) p(y | y)$. You are summing over all the possible values that $Y$ can take. Any value other than $y$ has conditional probabiltiy of 0, so you are left with $g(y)$ in the end. $\endgroup$ Apr 15 '21 at 5:10
  • $\begingroup$ @SOULed_Outt I understand that $E[X|Y]$ is a RV which is a function of Y and E[X|Y=y] is a value evaluated at Y=y. Expected value should be evaluated at all possible value of RV, that is why the sum seems to be reasonable for me. $\endgroup$
    – Hunglk
    Apr 15 '21 at 5:35
  • $\begingroup$ The earlier comment I made was very misleading so I deleted it (apologies). Take a look at what I posted as an answer and let me know if it clears anything up. $\endgroup$ Apr 15 '21 at 5:37
  • $\begingroup$ Since $E[X|Y]$ is a (measurable) function of $Y$, it is a fixed (deterministic) and known quantity given a realisation of $Y$. Hence, in the probabilistic universe where $Y$ is observed as $y$, it is a constant, equal to its expectation. $\endgroup$
    – Xi'an
    Apr 15 '21 at 8:04

Suppose, for example, that $Y$ could take values $1,2,$ or $3$ and we want to find $E[g(Y) | Y =3]$ for some nice function $g$. By defintion of conditional expectation we have \begin{align} E [g(Y) | Y=3] &= \sum_y g(y) \cdot P(Y=y | Y=3) \\ &= g(1) \cdot P(Y=1 | Y=3) + g(2) \cdot P(Y =2 | Y=3) + g(3) \cdot P(Y = 3 | Y = 3) \end{align}

Note that $P(Y = 1 | Y = 3) = 0$, $P(Y = 2 | Y = 3) = 0$, and that $P(Y=3 | Y=3)=1$ so $$ E [g(Y) | Y=3] = g(3) $$

I think your problem starts after you introduce the summation. $p(y|y)$ is not always equal to 1. It equals zero for any $y' \neq y$. In the summation, the $y$ in the first slot of $p(y|y)$ varies while the $y$ in the second slot is fixed (see like in the example above).

Look at what I have highlighted in red. The red $y$ serves as a dummy variable. I could replace it with any letter I want so long as I understand that it should represent all possible values that $Y$ can take. Let's say I use $a$ instead. \begin{align} E [g(Y) | Y=y] &= \sum_{\color{red}{y}} g(\color{red}{y}) \cdot P(Y = \color{red}{y} | Y = y) \\ &= \sum_{a} g(a) \cdot P(Y = {a} | Y = y). \end{align}

Notice that $P(Y = a | Y= y) = \begin{cases} 1 \text{ if } a = y \\ 0 \text{ if } a \neq y \end{cases}.$ Therefore $E [g(Y) | Y=y] = g(y)$

  • $\begingroup$ It means that the Y under conditional sign (|) will hold constant while the Y above conditional sign (|) will be evaluated at all possible value? I thought that Y should be evaluated at both. For example, if Y takes value in $[1,2,3]$, we have:$\sum_y g(y).P(Y=y|Y=y) = g(1).P(Y=1|Y=1)+g(2).P(Y=2|Y=2)+g(3).P(Y=3|Y=3) = g(1)+g(2)+g(3) = \sum_y g(y)$ $\endgroup$
    – Hunglk
    Apr 15 '21 at 5:59
  • $\begingroup$ Yes to your question. You should only evaluate the part above the conditional sign. The part under the conditional sign is held constant. $\endgroup$ Apr 15 '21 at 6:16
  • $\begingroup$ In this case, what is the different between $E[X|Y]$ and $E[X|Y=y]$? As my knowledge, The first term is rv and the second term is the value. $\endgroup$
    – Hunglk
    Apr 15 '21 at 6:24
  • $\begingroup$ Sometimes they are used interchangeably. I believe most texts on probability say that you should view $E[X | Y]$ as a random variable and you should view $E[X|Y=y]$ as a number that depends on $y$. $\endgroup$ Apr 15 '21 at 6:36
  • $\begingroup$ That's why I got confused. my knowledge is $E[g(Y)|Y]$ is a r.v, then, when we evaluated it, we should evaluated both g(Y) and Y. $\endgroup$
    – Hunglk
    Apr 15 '21 at 7:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.