# Iterated expectations and variances examples

Suppose we generate a random variable $$X$$ in the following way. First we flip a fair coin. If the coin is heads, take $$X$$ to have a $$Unif(0,1)$$ distribution. If the coin is tails, take $$X$$ to have a $$Unif(3,4)$$ distribution.

Find the mean and standard deviation of $$X$$.

This is my solution. I wanted to check if it's correct or if there's a better approach.

Let $$Y$$ denote the random variable that is $$1$$ if the coin lands on a head and $$0$$ otherwise Firstly $$\mathbb{E}(\mathbb{E}(X|Y)) = \mathbb{E}(X)$$

Thus $$\mathbb{E}(\mathbb{E}(X|Y)) = \frac{1}{2} \cdot \mathbb{E}(X|Y=0) + \frac{1}{2} \cdot \mathbb{E}(X|Y=1) = \frac{1}{2} \cdot \frac{1}{2} + \frac{1}{2} \cdot \frac{7}{2}=2$$

Secondly $$\mathbb{V}(X) = \mathbb{E}(\mathbb{V}(X|Y))+\mathbb{V}(\mathbb{E}(X|Y))$$

Now $$\mathbb{V}(X|Y = 0) = \mathbb{V}(X|Y=1) = \frac{1}{12}$$. Thus $$\mathbb{E}(\mathbb{V}(X|Y)) = \frac{1}{12}$$. Next calculating $$\mathbb{V}(\mathbb{E}(X|Y)) = \mathbb{E}(\mathbb{E}(X^2|Y)) - (\mathbb{E}(\mathbb{E}(X|Y)))^2 = (\frac{1}{2} \cdot \frac{1}{4} + \frac{49}{4} \cdot \frac{1}{2}) - (2)^2 = \frac{50}{8} - 4.$$

Your calculation is correct, and is a good way I think. One other approach might be just using the PDF of $$X$$, using uniform PDF, $$\Pi(x)$$: $$f_X(x)=\frac{1}{2}\Pi(x)+\frac{1}{2}\Pi(x-3)$$ Expected value can be fairly easy via both method, we just need $$E[X^2]$$: $$E[X^2]=\frac{1}{2}\int_0^{1}x^2dx+\frac{1}{2}\int_3^4x^2dx=\frac{4^3-3^3+1^3}{6}=\frac{19}{3}$$ which yields $$\operatorname{var}(X)=19/3-4=7/3$$, as yours.

Note: Add 1/12 to your final answer, since your answer is for $$V(E[X|Y])$$.

• wait we get different answers for variance $7/3 \neq 50/8-4$ where did i go wrong? – Iltl Apr 20 at 14:26
• @Iltl your answer is for V(E[X|Y]), if you add E[V(X|Y)] to that, they become equal. – gunes Apr 20 at 15:24
• Gotcha i forgot i didn't include it thanks! – Iltl Apr 20 at 15:25

There are generally two ways to approach these types of problems: by (1) Finding the second stage expectation $$E(X)$$ with the theorem of total expectation; or by (2) Finding the second stage expectation $$E(X)$$, using $$f_{X}(x)$$. These are equivalent methods, but you might find one easier to comprehend, so I present them both in detail below for $$E(X)$$. The approach is similar for $$Var(X)$$, so I exclude its presentation, but can update my answer if you really need it.

## Method (1) Finding the second stage expectation $$E(X)$$ with the theorem of total expectation

In this case, the Theorem of Total Expectation states that:

$$\begin{eqnarray*} E(X) & = & \sum_{y=0}^{1}E(X|Y=y)P(Y=y)\\ & = & \sum_{y=0}^{1}E(X|Y=y)f_{Y}(y) \end{eqnarray*}$$

So, we simply need to find the corresponding terms in the line above for $$y=0$$ and $$y=1$$. We are given the following:

$$\begin{eqnarray*} f_{Y}(y) & = & \begin{cases} \frac{1}{2} & \text{for}\,y=0\,(heads),\,1\,(tails)\\ 0 & \text{otherwise} \end{cases} \end{eqnarray*}$$

and

$$\begin{eqnarray*} f_{X|Y}(x|y) & = & \begin{cases} 1 & \text{for}\,3

Now, we simply need to obtain $$E(X|Y=y)$$ for each realization of $$y$$:

$$\begin{eqnarray*} E(X|Y=y) & = & \int_{-\infty}^{\infty}xf_{X|Y}(x|y)dx\\ & = & \begin{cases} \int_{3}^{4}x(1)dx & \text{for}\,y=0\\ \int_{0}^{1}x(1)dx & \text{for}\,y=1 \end{cases}\\ & = & \begin{cases} \left.\frac{x^{2}}{2}\right|_{x=3}^{x=4} & \text{for}\,y=0\\ \left.\frac{x^{2}}{2}\right|_{x=0}^{x=1} & \text{for}\,y=1 \end{cases}\\ & = & \begin{cases} \frac{7}{2} & \text{for}\,y=0\\ \frac{1}{2} & \text{for}\,y=1 \end{cases} \end{eqnarray*}$$

So, substituting each term into the Theorem of Total Expectation above yields:

$$\begin{eqnarray*} E(X) & = & \sum_{y=0}^{1}E(X|Y=y)f_{Y}(y)\\ & = & E(X|Y=0)f_{Y}(0)+E(X|Y=1)f_{Y}(1)\\ & = & \left(\frac{7}{2}\right)\left(\frac{1}{2}\right)+\left(\frac{1}{2}\right)\left(\frac{1}{2}\right)\\ & = & 2 \end{eqnarray*}$$

## Method (2) Finding the second stage expectation $$E(X)$$, using $$f_{X}(x)$$

To use this method, we first find the $$f_{X,Y}(x,y)$$ and $$f_{X}(X)$$. To begin, recall that $$f_{X,Y}(x,y)$$ is given by:

$$\begin{eqnarray*} f_{X,Y}(x,y) & = & f_{X|Y}(x|y)f_{Y}(y)\\ & = & \begin{cases} \left(1\right)\left(\frac{1}{2}\right) & \text{for}\,3

and we can find $$f_{X}(x)$$ by summing out the $$y$$ component:

$$\begin{eqnarray*} f_{X}(x) & = & \sum_{y=0}^{1}f_{X,Y}(x,y)\\ & = & f_{X,Y}(x,0)+f_{X,Y}(x,1)\\ & = & \frac{1}{2}I(3\le x\le4)+\frac{1}{2}I(0\le x\le1) \end{eqnarray*}$$

And now, we can just find $$E(X)$$ using the probability density function of $$f_{X}(x)$$ as usual:

$$\begin{eqnarray*} E(X) & = & \int_{-\infty}^{\infty}xf_{X}(x)dx\\ & = & \int_{-\infty}^{\infty}x\left[\frac{1}{2}I(3\le x\le4)+\frac{1}{2}I(0\le x\le1)\right]dx\\ & = & \frac{1}{2}\int_{-\infty}^{\infty}xI(3\le x\le4)dx+\frac{1}{2}\int_{-\infty}^{\infty}xI(0\le x\le1)dx\\ & = & \frac{1}{2}\int_{3}^{4}xdx+\frac{1}{2}\int_{0}^{1}xdx\\ & = & \left(\frac{1}{2}\right)\left.\left(\frac{x^{2}}{2}\right)\right|_{x=3}^{x=4}+\left(\frac{1}{2}\right)\left.\left(\frac{x^{2}}{2}\right)\right|_{x=0}^{x=1}\\ & = & \left(\frac{1}{2}\right)\left(\frac{7}{2}\right)+\left(\frac{1}{2}\right)\left(\frac{1}{2}\right)\\ & = & 2 \end{eqnarray*}$$

the same two approaches can be used to compute $$Var(X)$$.

This problem can be simplified substantially by decomposing the random variable of interest as a sum of two independent parts:

$$X = U+3V \quad \quad \quad \quad U \sim \text{U}(0,1) \quad \quad \quad \quad V \sim \text{Bern}(\tfrac{1}{2}).$$

Using this decomposition we have mean:

\begin{aligned} \mathbb{E}(X) = \mathbb{E}(U+3V) &= \mathbb{E}(U) + 3 \mathbb{E}(V) \\[6pt] &= \frac{1}{2} + 3 \cdot \frac{1}{2} = 2, \\[6pt] \end{aligned}

and variance:

\begin{aligned} \mathbb{V}(X) = \mathbb{V}(U+3V) &= \mathbb{V}(U) + 3^2 \mathbb{V}(V) \\[6pt] &= \frac{1}{12} + 9 \cdot \frac{1}{4} \\[6pt] &= \frac{1}{12} + \frac{27}{12} \\[6pt] &= \frac{28}{12} = \frac{7}{3}, \\[6pt] \end{aligned}

which gives the corresponding standard deviation:

\begin{aligned} \mathbb{S}(X) = \sqrt{\mathbb{V}(X)} &= \sqrt{\frac{7}{3}} \approx 1.527525. \\[6pt] \end{aligned}

As you can see, this simplifies the calculations substantially, and does not require the use of iterated expectations or variance.

Comment: Here is a brief simulation, comparing approximate simulated results with theoretical results derived in this Q and A. Everything below matches within the margin of simulation error.

Also see Wikipedia on Mixture Distributions, under Moments, for some relevant formulas.

set.seed(420)  # for reproducibility
u1 = runif(10^6);  u2 = runif(10^6, 3, 4)
ht = rbinom(10^6, 1, .5)
x = ht*u1 + (1-ht)*u2
mean(x);  2
[1] 2.001059   # aprx E(X) = 2
[1] 2          # proposed exact
var(x); 7/3
[1] 2.332478   # aprx Var(X)
[1] 2.333333
mean(x^2); 19/3
[1] 6.336712   # aprx E(X^2)
[1] 6.333333

hist(x, br=40, prob=T, col="skyblue2")