5
$\begingroup$

Consider two Poisson binomial distributions (distributions of sums of independent Bernoulli variables) as below:

  1. The distribution of the sum of 99 variables with probability 0.01 (of taking the value 1), and 1 variable with probability 0.05.
  2. The distribution of the sum of 999 variables with probability 0.01, and 1 variable with probability 0.05.

We expect that each distribution will approximate to a Poisson distribution:

$$Pr(k)=\frac{\lambda^ke^{-\lambda}}{k!}.......(k=0,1,2,...) $$

with $\lambda = 1.04$ for Distribution 1 and $\lambda = 10.04$ for Distribution 2.

Intuitively, to me at least, the approximation should be better for Distribution 2, because there are more variables with probability 0.01 to "swamp" the effect of the single variable with a different probability. However, applying Le Cam's Theorem, the sums to infinity of the absolute differences in probabilities of particular $k$-values under these distributions from their respective Poisson approximations have the following upper bounds:

$$Distribution 1: 2[(99*0.01^2)+(1*0.05^2)] = 0.0248$$

$$Distribution 2: 2[(999*0.01^2)+(1*0.05^2)] = 0.2048$$

If these sums can be taken as measures of goodness of approximation, they suggest that the approximation is much better for Distribution 1.

How can this puzzle be resolved? Is there any result that goes beyond Le Cam's Theorem by saying something about the absolute differences in probabilities of particular $k$-values, and not merely about their sum? It would be helpful for example if it could be shown that at most only a small part of the 0.2048 for Distribution 2 can relate to any particular $k$-value.

Update I calculated the differences between the exact probabilities of $k$-values and their Poisson approximations, obtaining the results below:

Distribution 1: Sum of absolute differences for $k=0,...,10 = 0.0066$. Largest absolute difference for a particular $k$ is $0.0022$ for $k=0$.

Distribution 2: Sum of absolute differences for $k=0,...,30 = 0.0050$. Largest absolute difference for a particular $k$ is $0.0006$ for $k=10$.

I did not calculate beyond 10 and 30 respectively as the probabilities then became very small and appeared unlikely to contribute much to the sums to infinity. These results suggest that the Poisson approximation is indeed better for Distribution 2. They also suggest that the upper bounds obtained from Le Cam's Theorem (though perfectly correct) do not necessarily provide useful measures of goodness of approximation because for particular distributions the actual sums of absolute differences may lie well within those bounds.

$\endgroup$
3
$\begingroup$

I think your contradiction is resolved if you forget about the p=0.05 variable for a second, and take a look at p=0.01 variables (this converts the Poisson binomial distributions into simple binomial distributions). In this case the distribution 1 will be approximated by Poisson distribution better than distribution 2. See this. In the second case $np\sim10$, i.e. right at the bound where Poisson is a good approximation to Binomial. In the first case $np=0.99$, i.e. much smaller number, hence better approximation.

So, it's not the p=0.05 variable that is a factor in goodness of fit in your examples.

$\endgroup$
  • $\begingroup$ This answer is helpful in suggesting a focus on the p=0.01 variables and on the conditions under which a binomial distribution is approximated by a Poisson distribution. However, I have not been able to confirm with sample calculations that the approximation becomes materially worse at around $np=10$. The source linked in the answer states this as a rule of thumb without either explaining its basis or quoting any source in turn. It would be useful to have a reference to a source which demonstrates this property of Poisson approximation by mathematical reasoning or numerical examples. $\endgroup$ – Adam Bailey Dec 17 '14 at 18:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.