Conditional vs unconditional expectation

I'm having trouble understanding the calculation of conditional versus unconditional expectations in this case:

\begin{align} ~ \\ ~ \\ ~ \\ E(Y)&=\sum_Y y f(y) \\ ~ \\ ~ \\ ~ \\ E(Y|X)&=\sum_Y y f(y|x) \end{align}

To me, both calculations are $1*0.25 + 1*0.25 + 2*0.5 = 1.5$. What am I doing wrong?

• The second calculation should give you a random variable rather than a number. Did you notice that the right hand side of the second formula depends on the unknown $x$? – whuber Aug 30 '13 at 20:46
• Yes, but I wasn't sure how to incorporate the x into the calculation, which asks for the value of y. I need to prove that E(Y) $\neq$ E(Y|X) – guest99 Aug 30 '13 at 20:51
• You are not using the correct values of $f(y\mid x)$. – Dilip Sarwate Aug 30 '13 at 21:08

As @Whuber said, second calculation should give you a random variable rather than a number.

Now to complete these calculations (and to see @Whubers fact) we calculate the following:

$$E(Y)=\sum_y yf(y)=1(0.25) + 1(0.25) + 2(0.5) = 1.5$$

which is what you correctly calculated before. Now, the second expectation is the following:

$$E(Y|X=x)=\sum_y y f(y|X=x)= \begin{cases} 1\times1 + 0\times 2=1 & \text{if } x = 1\\ 1\times 1 + 0\times 2=1 & \text{if } x = -1\\ 0\times 1 + 1\times2=2 & \text{if }x = 2 \end{cases}$$

So depending on what value $X$ takes determines what the expected value of $Y$ (conditioned $X$) will be.

• Er no, your calculation of $E(Y\mid X)$ is incorrect. Given $X = 1$, $Y$ is a degenerate random variable that takes on value $1$ with probability $1$ and so $$E(Y\mid X) = \begin{cases}1, &X=1,\\1, &X=-1,\\2, & X=2,\end{cases}$$ – Dilip Sarwate Aug 30 '13 at 21:12
• Thank you! In cases where E(Y)=E(Y|X), would both expectations be numbers? – guest99 Aug 30 '13 at 21:12
• @Dilip is right. One way to double check such a calculation is to verify that the expectation of $E(Y|X)$ equals the expectation of $Y$ itself, and indeed because $\Pr(X=1)=\Pr(X=-1)=1/4$ and $\Pr(X=2)=1/2,$ we compute the former expectation as $1/4\times 1+1/4\times 1 + 1/2\times 2 = 1.5$, agreeing with the calculation of the latter in the question. – whuber Aug 30 '13 at 21:18
• The expression "$E(Y)=E(Y|X)$" normally is nonsensical because the left hand side is a number and the right hand side is a random variable. – whuber Aug 30 '13 at 21:21
• But isn't this the definition of mean independence? – guest99 Aug 30 '13 at 21:21

If $p_{X,Y}(x,y)$ denotes the joint probability mass function of discrete random variables $X$ and $Y$, then the marginal mass functions are \begin{align} p_X(x) &= \sum_y p_{X,Y}(x,y)\\ p_Y(y) &= \sum_x p_{X,Y}(x,y) \end{align} and so we have that $$E[Y] = \sum_y y\cdot p_{Y}(y) = \sum_y y\cdot \sum_xp_{X,Y}(x,y) = \sum_x\sum_y y\cdot p_{X,Y}(x,y).\tag{1}$$

Now, the conditional probability mass function of $Y$ given that $X = x$ is $$p_{Y\mid X}(y \mid X=x) = \frac{p_{X,Y}(x,y)}{p_X(x)} = \frac{p_{X,Y}(x,y)}{\sum_y p_{X,Y}(x,y)}\tag{2}$$ and $$E[Y\mid X=x] = \sum_y y\cdot p_{Y\mid X}(y \mid X=x).\tag{3}$$ The value of this expectation depends on our choice of the value $x$ taken on by $X$ and is thus a random variable; indeed, it is a function of the random variable $X$, and this random variable is denoted $E[Y\mid X]$. It happens to take on values $E[Y\mid X = x_1], E[Y\mid X=x_2], \cdots$ with probabilities $p_X(x_1), p_X(x_2), \cdots$ and so its expected value is

\begin{align}E\bigr[E[Y\mid X]\bigr] &= \sum_x E[Y\mid X = x]\cdot p_X(x) &\text{note the sum is w.r.t}~x\\ &= \sum_x \left[\sum_y y\cdot p_{Y\mid X}(y \mid X=x)\right]\cdot p_X(x) &\text{using}~ (3)\\ &= \sum_x \left[\sum_y y\cdot \frac{p_{X,Y}(x,y)}{p_X(x)}\right]\cdot p_X(x) &\text{using}~ (2)\\ &= \sum_x \sum_y y\cdot p_{X,Y}(x,y)\\ &= E[Y] &\text{using}~(1) \end{align}

In general, the number $E[Y\mid X = x]$ need not equal the number $E[Y]$ for any $x$. But, if $X$ and $Y$ are independent random variables and so $p_{X,Y}(x,y) = p_X(x)p_Y(y)$ for all $x$ and $y$, then $$p_{Y\mid X}(y \mid X=x) = \frac{p_{X,Y}(x,y)}{p_X(x)} = \frac{p_X(x)p_Y(y)}{p_X(x)} = p_Y(y)\tag{4}$$ and so $(3)$ gives $$E[Y\mid X=x] = \sum_y y\cdot p_{Y\mid X}(y \mid X=x) = \sum_y y\cdot p_Y(y) = E[Y]$$ for all $x$, that is, $E[Y\mid X]$ is a degenerate random variable that equals the number $E[Y]$ with probability $1$.

In your particular example, BabakP's answer after correction by Moderator whuber shows that $E[Y\mid X = x]$ is a random variable that takes on values $1, 1, 2$ with probabilities $0.25, 0.25, 0.5$ respectively and so its expectation is $0.25\times 1 + 0.25\times 1 + 0.5\times 2 = 1.5$ while the $Y$ itself is a random variable taking on values $1$ and $1$ with equal probability $0.5$ and so $E[Y] = 1\times 0.5 + 2\times 0.5 = 1.5$ as indeed one expects from the law of iterated expectation $$E\left[[Y\mid X]\right] = E[Y].$$

If the joint pmf was intended to illustrate the difference between conditional expectation and expectation, then it was a spectacularly bad choice because the random variable $E[Y\mid X]$ turns out to have the same distribution as the random variable $Y$, and so the expected values are necessarily the same. More generally, $E[Y\mid X]$ does not have the same distribution as $Y$ but their expected values are the same. Consider for exampple, the joint pmf $$\begin{array}{c c c c} & \quad &X=1\quad &X=-1\quad &X=2 \\ &Y=1\quad &0.2\quad &0.2\quad &0.1 \\ &Y=2\quad &0.2\quad &0.1\quad &0.2 \end{array}$$ for which the conditional pmfs of $Y$ are $$X=1: \qquad p_{Y\mid X}(1\mid X = 1) = \frac{1}{2}, p_{Y\mid X}(2\mid X = 1) = \frac{1}{2}\\ X=-1: \qquad p_{Y\mid X}(1\mid X = 1) = \frac{2}{3}, p_{Y\mid X}(2\mid X = 1) = \frac{1}{3}\\ X=2: \qquad p_{Y\mid X}(1\mid X = 1) = \frac{1}{3}, p_{Y\mid X}(2\mid X = 1) = \frac{2}{3}$$ the conditional means are \begin{align} E[Y\mid X = 1] &= 1\times \frac{1}{2} + 2 \times \frac{1}{2} = \frac{3}{2}\\ E[Y\mid X = -1] &= 1\times \frac{2}{3} + 2 \times \frac{1}{3} = \frac{4}{3}\\ E[Y\mid X = 2] &= 1\times \frac{1}{3} + 2 \times \frac{2}{3} = \frac{5}{3} \end{align} that is, $E[Y\mid X]$ is a random variable that takes on values $\frac{3}{2}, \frac{4}{3}, \frac{5}{3}$ with probabiliities $\frac{4}{10}, \frac{3}{10}, \frac{3}{10}$ respectively which is not the same as the distribution of $Y$. Note also that $E[Y] = \frac{3}{2}$ happens to equal $E[Y\mid X=1]$ but not the other two conditional expectations. While $E[Y\mid X]$ and $Y$ have different distributions, their expected values are the same: $$E\left[E[Y\mid X]\right] = \frac{3}{2}\times\frac{4}{10} +\frac{4}{3}\times\frac{3}{10} + \frac{5}{3}\times \frac{3}{10} = \frac{3}{2} = E[Y].$$