I've written some code to calculate Poisson confidence limits (a) using Chi-squared, and (b) from first principles, using Poisson's probability mass function equation. However, the two sets of results don't agree. For example, for Lambda=10 and 95% confidence limits, I get:
95%: Chi squared [4.80, 18.39] Exact [3, 17]
The discrepancy is worse for wider confidence limits:
3 sigma: Chi squared [3.08, 23.64] Exact [1, 21]
Note that my 'exact' result shows the first k outside the confidence limits (in other words, the range [4,16] is completely inside the 95% range).
My 95% Chi squared result above agrees with several online Poisson limits calculators. Statpages, for example, also gives [4.80, 18.39].
However, the exact result also appears to be correct. For 95%, the online calculator at Stattrek appears to give the same results as my exact [3,17]. More precisely, here are cumulative probabilities for lambda=10 taken from a table here:
0 - 2 events: negligible 3 1.0% 4 2.9% ... 15 95.1% 16 97.3% 17 98.6%
So, the 95 confidence limits are for 4 to 16 events, inclusive, which agrees with my program output, which says that <= 3 events, or >= 17 events, are outside the 95% limits.
Have I got this wrong somewhere? Is it just that Lambda=10 is too small for the chi-squared method to be exact? If I increase Lambda to 100 I get:
95%: Chi squared [81.36, 121.63] Exact [80, 120] 3 sigma: Chi squared [72.65, 133.83] Exact [70, 131]
It essentially makes no difference. I can live with the fact that the inexact result is continuous, but not with the inaccuracy.
Thanks for the comments, everyone. As I understand it, the basic answer is that they both provide confidence limits, but they're different, and I shouldn't expect them to be the same, and should just live with it - correct?
For background, this is for analysing healthcare providers, and finding out if any differ significantly from the average. The important thing here (for me, anyway) is not to point the finger at somebody and say that they're outside the 2 or 3 SD limits, when another analysis could show that they're actually inside the limits. For the same reason, I don't care that a discrete method doesn't give me exact 95% coverage - I just need to positively identify outliers.
My own background isn't stats, but I do understand the exact method, and I'm happy that it gives the "right" answer (notwithstanding the fact that the processes aren't really appropriate for Poisson). However, I don't understand the Poisson/Chi-squared transformation, and I'm not happy with it for this application, because it 'incorrectly' adds outliers at the low end of the range (not to mention missing 'real' outliers at the top). However, it is universally used for exactly this application. Would it be fair for me to say that the exact method is better for this application, and the approximation is simply that, and it is incorrect?