7
$\begingroup$

I'm having trouble understanding the calculation of conditional versus unconditional expectations in this case:

\begin{array}{c c c c} & \quad &X=1\quad &X=-1\quad &X=2 \\ &Y=1\quad &0.25\quad &0.25\quad &0 \\ &Y=2\quad &0\quad &0\quad &0.5 \end{array}

\begin{align} ~ \\ ~ \\ ~ \\ E(Y)&=\sum_Y y f(y) \\ ~ \\ ~ \\ ~ \\ E(Y|X)&=\sum_Y y f(y|x) \end{align}

To me, both calculations are $1*0.25 + 1*0.25 + 2*0.5 = 1.5$. What am I doing wrong?

$\endgroup$
  • $\begingroup$ The second calculation should give you a random variable rather than a number. Did you notice that the right hand side of the second formula depends on the unknown $x$? $\endgroup$ – whuber Aug 30 '13 at 20:46
  • $\begingroup$ Yes, but I wasn't sure how to incorporate the x into the calculation, which asks for the value of y. I need to prove that E(Y) $\neq$ E(Y|X) $\endgroup$ – guest99 Aug 30 '13 at 20:51
  • $\begingroup$ You are not using the correct values of $f(y\mid x)$. $\endgroup$ – Dilip Sarwate Aug 30 '13 at 21:08
4
$\begingroup$

As @Whuber said, second calculation should give you a random variable rather than a number.

Now to complete these calculations (and to see @Whubers fact) we calculate the following:

$$E(Y)=\sum_y yf(y)=1(0.25) + 1(0.25) + 2(0.5) = 1.5$$

which is what you correctly calculated before. Now, the second expectation is the following:

$$E(Y|X=x)=\sum_y y f(y|X=x)= \begin{cases} 1\times1 + 0\times 2=1 & \text{if } x = 1\\ 1\times 1 + 0\times 2=1 & \text{if } x = -1\\ 0\times 1 + 1\times2=2 & \text{if }x = 2 \end{cases}$$

So depending on what value $X$ takes determines what the expected value of $Y$ (conditioned $X$) will be.

$\endgroup$
  • 1
    $\begingroup$ Er no, your calculation of $E(Y\mid X)$ is incorrect. Given $X = 1$, $Y$ is a degenerate random variable that takes on value $1$ with probability $1$ and so $$E(Y\mid X) = \begin{cases}1, &X=1,\\1, &X=-1,\\2, & X=2,\end{cases}$$ $\endgroup$ – Dilip Sarwate Aug 30 '13 at 21:12
  • $\begingroup$ Thank you! In cases where E(Y)=E(Y|X), would both expectations be numbers? $\endgroup$ – guest99 Aug 30 '13 at 21:12
  • $\begingroup$ @Dilip is right. One way to double check such a calculation is to verify that the expectation of $E(Y|X)$ equals the expectation of $Y$ itself, and indeed because $\Pr(X=1)=\Pr(X=-1)=1/4$ and $\Pr(X=2)=1/2,$ we compute the former expectation as $1/4\times 1+1/4\times 1 + 1/2\times 2 = 1.5$, agreeing with the calculation of the latter in the question. $\endgroup$ – whuber Aug 30 '13 at 21:18
  • $\begingroup$ The expression "$E(Y)=E(Y|X)$" normally is nonsensical because the left hand side is a number and the right hand side is a random variable. $\endgroup$ – whuber Aug 30 '13 at 21:21
  • $\begingroup$ But isn't this the definition of mean independence? $\endgroup$ – guest99 Aug 30 '13 at 21:21
4
$\begingroup$

If $p_{X,Y}(x,y)$ denotes the joint probability mass function of discrete random variables $X$ and $Y$, then the marginal mass functions are $$\begin{align} p_X(x) &= \sum_y p_{X,Y}(x,y)\\ p_Y(y) &= \sum_x p_{X,Y}(x,y) \end{align}$$ and so we have that $$E[Y] = \sum_y y\cdot p_{Y}(y) = \sum_y y\cdot \sum_xp_{X,Y}(x,y) = \sum_x\sum_y y\cdot p_{X,Y}(x,y).\tag{1}$$

Now, the conditional probability mass function of $Y$ given that $X = x$ is $$p_{Y\mid X}(y \mid X=x) = \frac{p_{X,Y}(x,y)}{p_X(x)} = \frac{p_{X,Y}(x,y)}{\sum_y p_{X,Y}(x,y)}\tag{2}$$ and $$E[Y\mid X=x] = \sum_y y\cdot p_{Y\mid X}(y \mid X=x).\tag{3}$$ The value of this expectation depends on our choice of the value $x$ taken on by $X$ and is thus a random variable; indeed, it is a function of the random variable $X$, and this random variable is denoted $E[Y\mid X]$. It happens to take on values $E[Y\mid X = x_1], E[Y\mid X=x_2], \cdots $ with probabilities $p_X(x_1), p_X(x_2), \cdots$ and so its expected value is

$$\begin{align}E\bigr[E[Y\mid X]\bigr] &= \sum_x E[Y\mid X = x]\cdot p_X(x) &\text{note the sum is w.r.t}~x\\ &= \sum_x \left[\sum_y y\cdot p_{Y\mid X}(y \mid X=x)\right]\cdot p_X(x) &\text{using}~ (3)\\ &= \sum_x \left[\sum_y y\cdot \frac{p_{X,Y}(x,y)}{p_X(x)}\right]\cdot p_X(x) &\text{using}~ (2)\\ &= \sum_x \sum_y y\cdot p_{X,Y}(x,y)\\ &= E[Y] &\text{using}~(1) \end{align}$$


In general, the number $E[Y\mid X = x]$ need not equal the number $E[Y]$ for any $x$. But, if $X$ and $Y$ are independent random variables and so $p_{X,Y}(x,y) = p_X(x)p_Y(y)$ for all $x$ and $y$, then $$p_{Y\mid X}(y \mid X=x) = \frac{p_{X,Y}(x,y)}{p_X(x)} = \frac{p_X(x)p_Y(y)}{p_X(x)} = p_Y(y)\tag{4}$$ and so $(3)$ gives $$E[Y\mid X=x] = \sum_y y\cdot p_{Y\mid X}(y \mid X=x) = \sum_y y\cdot p_Y(y) = E[Y]$$ for all $x$, that is, $E[Y\mid X]$ is a degenerate random variable that equals the number $E[Y]$ with probability $1$.


In your particular example, BabakP's answer after correction by Moderator whuber shows that $E[Y\mid X = x]$ is a random variable that takes on values $1, 1, 2$ with probabilities $0.25, 0.25, 0.5$ respectively and so its expectation is $0.25\times 1 + 0.25\times 1 + 0.5\times 2 = 1.5$ while the $Y$ itself is a random variable taking on values $1$ and $1$ with equal probability $0.5$ and so $E[Y] = 1\times 0.5 + 2\times 0.5 = 1.5$ as indeed one expects from the law of iterated expectation $$E\left[[Y\mid X]\right] = E[Y].$$

If the joint pmf was intended to illustrate the difference between conditional expectation and expectation, then it was a spectacularly bad choice because the random variable $E[Y\mid X]$ turns out to have the same distribution as the random variable $Y$, and so the expected values are necessarily the same. More generally, $E[Y\mid X]$ does not have the same distribution as $Y$ but their expected values are the same. Consider for exampple, the joint pmf $$\begin{array}{c c c c} & \quad &X=1\quad &X=-1\quad &X=2 \\ &Y=1\quad &0.2\quad &0.2\quad &0.1 \\ &Y=2\quad &0.2\quad &0.1\quad &0.2 \end{array}$$ for which the conditional pmfs of $Y$ are $$X=1: \qquad p_{Y\mid X}(1\mid X = 1) = \frac{1}{2}, p_{Y\mid X}(2\mid X = 1) = \frac{1}{2}\\ X=-1: \qquad p_{Y\mid X}(1\mid X = 1) = \frac{2}{3}, p_{Y\mid X}(2\mid X = 1) = \frac{1}{3}\\ X=2: \qquad p_{Y\mid X}(1\mid X = 1) = \frac{1}{3}, p_{Y\mid X}(2\mid X = 1) = \frac{2}{3}$$ the conditional means are $$\begin{align} E[Y\mid X = 1] &= 1\times \frac{1}{2} + 2 \times \frac{1}{2} = \frac{3}{2}\\ E[Y\mid X = -1] &= 1\times \frac{2}{3} + 2 \times \frac{1}{3} = \frac{4}{3}\\ E[Y\mid X = 2] &= 1\times \frac{1}{3} + 2 \times \frac{2}{3} = \frac{5}{3} \end{align}$$ that is, $E[Y\mid X]$ is a random variable that takes on values $\frac{3}{2}, \frac{4}{3}, \frac{5}{3}$ with probabiliities $\frac{4}{10}, \frac{3}{10}, \frac{3}{10}$ respectively which is not the same as the distribution of $Y$. Note also that $E[Y] = \frac{3}{2}$ happens to equal $E[Y\mid X=1]$ but not the other two conditional expectations. While $E[Y\mid X]$ and $Y$ have different distributions, their expected values are the same: $$E\left[E[Y\mid X]\right] = \frac{3}{2}\times\frac{4}{10} +\frac{4}{3}\times\frac{3}{10} + \frac{5}{3}\times \frac{3}{10} = \frac{3}{2} = E[Y].$$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.