Comparing Proportions - Mix of Small and Large Sample Sizes

Question

I have twenty groups of observations (A1, A2, ... A20), each includes positive and negative cases. I would like to test whether the difference between proportions of positives of each group to others is statistically significant. However, there are a few concerns which I am not sure how to deal with.

Firstly, how can I compare p1 (proportion of positives in group A1) for example to the rest? I know the statistical test is not valid if I compare p1 to p_all where p_all is the proportion of positives in all groups combined (A_all = A1 U A2 U ... U A20), because A1 and A_all are not independent. In fact, A1 is a subset of A_all. One option would be to compare p1 with p1_rest where p1_rest is proportion of positives in all other nineteen groups combined (A1_rest = A2 U A3 U ... U A20). Is this correct?

Secondly, there is a mix of large and small sizes. Here are a few case comparisons:

1. for example, comparing p1 to p1_rest where sample sizes are |A1| = 10 and |A1_rest| = 40,000,
2. comparing p10 to p10_rest where sample sizes are |A10| = 10,000 and |A10_rest| = 30,010.

Lastly, the proportion of positives are high in each group (between 70% to 100%), and hence, the proportion of negatives are close to zero in some cases.

Any idea about comparing each group to others and the type of test I can use?

Answer 1

It would help if you could clarify your ultimate inference goal, maybe tell us the concrete context. Failing that, you seem to have data which can be presented as a contingency table with, say, 2 rows and 20 columns (plus the margins.) So your twenty groups are the columns, count of positives in first row, count of negatives in second row. Then, as a start a test of the null hypothesis of homegeneity, that is all the $p_i$'s are equal, is given by the chisquare test. Except that some small counts might make that a bad approximation. One way of getting confidence intervals for the individual $p$'s is via logistic regression.

For more specific ideas we need some detailed context!