Testing small proportions My data is made up of 4 groups, each with a large and identical sample size of 20,000.  The number of successes in each group is fairly small:
Group             A          B           C          D
Sample         20,000     20,000      20,000     20,000
Successes       90         120         110         200
Pr(Success)     0.5%       0.6%        0.6%        1.0%

I've done a simple chi-square test and found a significant difference (hope that was right!).  Is it now appropriate to then do t-tests or are the proportions too small and I should do a series of further chi-squared tests? (I'm generally unsure about if/ when t-test is appropriate for very small. large proportions even if np and nq >20).
Thanks!
 A: *

*RE: suitable test. I would use the Poisson distribution to model this situation rather than the binomial, since n is large and p is small. The null hypothesis is that all 4 rate parameters are the same vs alternative that at least one differs from the rest.

*Having done this test, and if I got a significant result, I could then study which groups are different. You could do this simply by quoting the rates with their confidence intervals. 

*I assume that your "t-test" are supposed to do pair-wise comparisons of the means in the presence of a significant result for the overall test. In the normal variate case, post-hoc comparisons are handled through things like Tukey's HSD or a Bonferroni adjustment. Long story short: you need to do your pair-wise comparisons with a (much) smaller p-value than you would normally use, to compensate for the fact that you are doing many tests. 

*I don't actually like post-hoc comparisons since it's not clear what the true p-values are,even with adjustments. Since you only do the tests conditionally on having obtained a significant result earlier on, the true p-value depends on a rather complicated distribution -- not the usual null hypothesis distribution.


To me, all the information your study has to offer can be expressed in the overall test of equality of the means, plus quoted means and confidence intervals. If a particular comparison interests you, you could also quote a confidence interval for the difference in the rates (or their ratio, since we're dealing with Poisson variates).
