Consider two Poisson binomial distributions (distributions of sums of independent Bernoulli variables) as below:
- The distribution of the sum of 99 variables with probability 0.01 (of taking the value 1), and 1 variable with probability 0.05.
- The distribution of the sum of 999 variables with probability 0.01, and 1 variable with probability 0.05.
We expect that each distribution will approximate to a Poisson distribution:
with $\lambda = 1.04$ for Distribution 1 and $\lambda = 10.04$ for Distribution 2.
Intuitively, to me at least, the approximation should be better for Distribution 2, because there are more variables with probability 0.01 to "swamp" the effect of the single variable with a different probability. However, applying Le Cam's Theorem, the sums to infinity of the absolute differences in probabilities of particular $k$-values under these distributions from their respective Poisson approximations have the following upper bounds:
$$Distribution 1: 2[(99*0.01^2)+(1*0.05^2)] = 0.0248$$
$$Distribution 2: 2[(999*0.01^2)+(1*0.05^2)] = 0.2048$$
If these sums can be taken as measures of goodness of approximation, they suggest that the approximation is much better for Distribution 1.
How can this puzzle be resolved? Is there any result that goes beyond Le Cam's Theorem by saying something about the absolute differences in probabilities of particular $k$-values, and not merely about their sum? It would be helpful for example if it could be shown that at most only a small part of the 0.2048 for Distribution 2 can relate to any particular $k$-value.
Update I calculated the differences between the exact probabilities of $k$-values and their Poisson approximations, obtaining the results below:
Distribution 1: Sum of absolute differences for $k=0,...,10 = 0.0066$. Largest absolute difference for a particular $k$ is $0.0022$ for $k=0$.
Distribution 2: Sum of absolute differences for $k=0,...,30 = 0.0050$. Largest absolute difference for a particular $k$ is $0.0006$ for $k=10$.
I did not calculate beyond 10 and 30 respectively as the probabilities then became very small and appeared unlikely to contribute much to the sums to infinity. These results suggest that the Poisson approximation is indeed better for Distribution 2. They also suggest that the upper bounds obtained from Le Cam's Theorem (though perfectly correct) do not necessarily provide useful measures of goodness of approximation because for particular distributions the actual sums of absolute differences may lie well within those bounds.