Say the success probability $X$ is a random variable with mean $\mu$ and Variance $\sigma^2$ which takes values in $[0,1]$. How can I compute the variance of a random Variable $Y$ which is 1 with probability $X$ and 0 with probability $1-X$. So there are "two layers" of variance.

  • $\begingroup$ Is this a homework or practice problem? It sounds like you're already given the variance. $\endgroup$ – Arya McCarthy May 16 at 15:54
  • 1
    $\begingroup$ It is just something that I want to understand myself. I mean that there is now another random variable $Y$ which is 1 with probability $X$ (which is random itself) and 0 with probability $1- X$. So there are "two layers" of variance and I dont know how to deal with this. $\endgroup$ – jonaden May 16 at 16:00
  • $\begingroup$ Hi, please edit important details like that into the question itself. $\endgroup$ – Arya McCarthy May 16 at 16:02

In general, you solve problems like this using the 'law of iterated variance'.

Let $Y|X \sim \text{Bern}(X)$ and use your stipulated prior mean and variance for the success probability $X$. Using the law of iterated variance, you get:

$$\begin{align} \mathbb{V}(Y) &= \mathbb{V}(\mathbb{E}(Y|X)) + \mathbb{E}(\mathbb{V}(Y|X)) \\[6pt] &= \mathbb{V}(X) + \mathbb{E}(X(1-X)) \\[6pt] &= \mathbb{V}(X) + \mathbb{E}(X) - \mathbb{E}(X^2) \\[6pt] &= \mathbb{E}(X^2) - \mathbb{E}(X)^2 + \mathbb{E}(X) - \mathbb{E}(X^2) \\[6pt] &= \mathbb{E}(X) - \mathbb{E}(X)^2 \\[6pt] &= \mu(1-\mu). \\[6pt] \end{align}$$

As you can see, the variance of $Y$ is determined by $\mu$, and is unaffected by $\sigma$. It turns out that this is true for all the moments of $Y$. In fact, we have $\mathbb{E}(f(Y)) = f(0) + [f(1)-f(0)] \mu$, so the expectation of any function of $Y$ is determined by $\mu$ and is unaffected by $\sigma$.

  • 1
    $\begingroup$ Thank you. This answers my question perfectly! However, I think there is a slight mistake in the second to last row, which also changes the result. $\endgroup$ – jonaden May 16 at 23:19
  • $\begingroup$ @Ben Surely, higher-order moments of $Y$ are not affected by $\sigma^2$ when $Y|X\sim \operatorname{Bern}(X)$ since the marginal point mass function of $Y$ can not be anything other than that of a Bernoulli random variable. $\endgroup$ – Jarle Tufto May 17 at 7:44
  • $\begingroup$ The higher-order moments still depend on $X$, and $\sigma$ affects the distribution of $X$, so I would expect it to affect the higher-order moments. $\endgroup$ – Ben May 17 at 9:11
  • $\begingroup$ @Ben No, the marginal distribution of $Y$ is clearly a Bernoulli distribution. Once its mean $EY=E(EY|X)=EX=\mu$ is given, the two point masses can only be $P(Y=0)=1-\mu$ and $P(Y=1)=\mu$ and the higher order moments are the same as that of any ordinary Bernoulli distribution and do not depend on $\sigma^2$. $\endgroup$ – Jarle Tufto May 17 at 9:35
  • 1
    $\begingroup$ Okay, yes, I see what you mean. That is correct. I have edited the answer to revise this part. $\endgroup$ – Ben May 17 at 10:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.