comparing proportions of several groups

Question

I have data of existence of a feature in n samples represents m groups. I want to find features with significantly different proportion in a group. I was thinking about the followings, and will be glad to get your input.

1) testing the proportions in each group using proportion test. The expected proportion of group i will be the relative frequency of samples in group i (# of sample in i/ all samples(n)). The observed will be the number of samples in group i in which the feature exists relative to all samples with the feature (from all groups). So if I have 100 samples, 20 in group i, in 10 samples of group i the feature exists out of 30 samples totally. The expected will be 20/100 and the observed will be 10/30.

2) Testing using contingency table.

• 2x2 table for each group in which I'll check the group i relative to all the other groups together. So the cells in the table will be existence in group i, existence in all other group, not exist in group i not exist in all groups. Back to the mention example- the table will look like:
• 2xm (m is number of groups) contingency table.

For the 2x2 I'll prefer using fisher exact test for its accuracy (and it is not limited by sample size). For the 2xm I think to use chi-square since it can be followed by Marascuillo procedure, unless there is a similar alternative for checking pairs of proportions followed by fisher.

I'll be very thankful for any input!

Answer 1

1) According to your description, your $2 \times 2$ table is correctly constructed. The proportion of Yes's in Gp 1 is $\hat p_1 = 10/30 = 0.333$ and your proportion of Yes's in the remaining 4 groups is $\hat p_R = 10/80 = 0.167.$ While these proportions are different, they are not 'significantly' different at the 5% level. Here are results from Minitab statistical software. The first test, based on a normal approximation gives a P-value of 0.073. Fisher's exact test gives a P-value of 0.106. Both P-values exceed 5%.

Test and CI for Two Proportions 

Sample   X   N  Sample p
1       10  30  0.333333
2       10  60  0.166667

Difference = p (1) - p (2)
Estimate for difference:  0.166667
95% CI for difference:  (-0.0265884, 0.359922)
Test for difference = 0 (vs ≠ 0):  Z = 1.79  P-Value = 0.073

Fisher’s exact test: P-Value = 0.106

(2) For illustration, I filled out some counts for the other four groups and did a chi-squared test to see if (according to my choices) the proportion of Yes's is the essentially the same in all groups (null hypothesis) or not. The null hypothesis was not rejected.

Chi-Square Test: Group, Yes No 

Rows: Group   Columns: Yes No

          Yes      No  All

1          10      10   20
        5.714  14.286
       3.2143  1.2857

2           4      16   20
        5.714  14.286
       0.5143  0.2057

3           3      15   18
        5.143  12.857
       0.8929  0.3571

4           4      16   20
        5.714  14.286
       0.5143  0.2057

5           7      13   20
        5.714  14.286
       0.2893  0.1157

All        28      70   98

Cell Contents:      Count
                    Expected count
                    Contribution to Chi-square


Pearson Chi-Square = 7.595, DF = 4, P-Value = 0.108

The second number in each of the ten cells is the expected count. All of them exceed 5, so the chi-squared statistic 7.595 can be properly compared with a chi-squared distribution with 4 degrees of freedom. The P-value 0.018 indicates that there are no differences among groups.

If the P-value were less than 0.05, then some ad hoc tests would be appropriate to see which groups had significant different probabilities of 'Yes' than others. A quick way to look for interesting results is to look at the third entry in each cell: 'Contributions to chi-squared'. If the overall P-value finds significant differences, then the 'Contributions' larger than 4 might point the way to them.

Also, you mentioned the Marascuillo procedure for formal ad hoc tests.