Consider a sequence of $n$ independent Bernoulli trials drawn from a list of biases $p_1,p_2,...,p_n\in[0,1]$, respectively. We set the random variable $X$ to be the sum of these trials. On wikipedia, the distribution of $X$ is called the Poisson binomial distribution. We define the sample mean and sample variance of our list of Bernoulli biases as $$ \bar{p}=\frac{1}{N}\sum_{i=1}^n p_i $$ and $$ \sigma_p^2 =\frac{1}{N}\sum_{i=1}^N(p_i-\bar{p}) =\frac{1}{N}\sum_{i=1}^N p_i^2 - \bar{p}^2. $$

Since the trials are independent, it is easy to compute that $$ \mathbb{E}[X] = \sum_{i=1}^n p_i = N\bar{p} $$ and \begin{align*} \mathbb{Var}[X] &= \sum_{i=1}^n p_i(1-p_i) \\ &= N\bar{p} - N(\sigma_p^2+\bar{p}^2) \\ &= N\bar{p}(1-\bar{p}) - N\sigma_p^2. \end{align*}

The expectation value of $X$ is not surprising. Also, when $\sigma_p^2=0$ we must have $\bar{p}=p_1=\cdots=p_n$ and so $X$ is binomially distributed, which matches $\mathbb{Var}[X]$ computed above.

My confusion is this: why does the variance of $X$ go down as the sample variance $\sigma_p^2$ goes up (with $\bar{p}$ and $N$ fixed)? I find this very counter-intuitive, and would appreciate an explanation. I would expect with a greater variance of biases, there would be a broader distribution of possible sums of the result...


Think of the case where $n=2$. If $p_1 = p_2 = 0.5$, this maximizes the variance of X. If $p_1 = 0$ and $p_2 = 1$, then $X=1$ and there is no variance.

| cite | improve this answer | |
  • $\begingroup$ Interesting. So you are saying, I think, that: the closer a given $p$ is to either 0 or 1, the more you know about the outcome of that trial. If $\sigma_p^2$ goes up, other quantities being fixed, then some of the $p$ values will be closer to either 0 or 1 (and somehow the ones that moved closer to $p=1/2$ have less of an effect than those that moved away from it?). Therefore you should be more certain about the total of the trials. $\endgroup$ – Ian Hincks Sep 30 '15 at 22:47
  • $\begingroup$ That is precisely the idea. You should be able to establish this by calculus, but since you had a formula that felt against your intuition I wanted to give you something simple. $\endgroup$ – jlimahaverford Sep 30 '15 at 23:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.