1
$\begingroup$

I have data of existence of a feature in n samples represents m groups. I want to find features with significantly different proportion in a group. I was thinking about the followings, and will be glad to get your input.

1) testing the proportions in each group using proportion test. The expected proportion of group i will be the relative frequency of samples in group i (# of sample in i/ all samples(n)). The observed will be the number of samples in group i in which the feature exists relative to all samples with the feature (from all groups). So if I have 100 samples, 20 in group i, in 10 samples of group i the feature exists out of 30 samples totally. The expected will be 20/100 and the observed will be 10/30.

2) Testing using contingency table.

  • 2x2 table for each group in which I'll check the group i relative to all the other groups together. So the cells in the table will be existence in group i, existence in all other group, not exist in group i not exist in all groups. Back to the mention example- the table will look like: enter image description here
  • 2xm (m is number of groups) contingency table.

enter image description here

For the 2x2 I'll prefer using fisher exact test for its accuracy (and it is not limited by sample size). For the 2xm I think to use chi-square since it can be followed by Marascuillo procedure, unless there is a similar alternative for checking pairs of proportions followed by fisher.

I'll be very thankful for any input!

$\endgroup$
1
  • $\begingroup$ You seem to be on the right track. I illustrated some of the possible tests in my Answer. $\endgroup$
    – BruceET
    Commented Aug 17, 2019 at 21:58

1 Answer 1

1
$\begingroup$

1) According to your description, your $2 \times 2$ table is correctly constructed. The proportion of Yes's in Gp 1 is $\hat p_1 = 10/30 = 0.333$ and your proportion of Yes's in the remaining 4 groups is $\hat p_R = 10/80 = 0.167.$ While these proportions are different, they are not 'significantly' different at the 5% level. Here are results from Minitab statistical software. The first test, based on a normal approximation gives a P-value of 0.073. Fisher's exact test gives a P-value of 0.106. Both P-values exceed 5%.

Test and CI for Two Proportions 

Sample   X   N  Sample p
1       10  30  0.333333
2       10  60  0.166667

Difference = p (1) - p (2)
Estimate for difference:  0.166667
95% CI for difference:  (-0.0265884, 0.359922)
Test for difference = 0 (vs ≠ 0):  Z = 1.79  P-Value = 0.073

Fisher’s exact test: P-Value = 0.106

(2) For illustration, I filled out some counts for the other four groups and did a chi-squared test to see if (according to my choices) the proportion of Yes's is the essentially the same in all groups (null hypothesis) or not. The null hypothesis was not rejected.

Chi-Square Test: Group, Yes No 

Rows: Group   Columns: Yes No

          Yes      No  All

1          10      10   20
        5.714  14.286
       3.2143  1.2857

2           4      16   20
        5.714  14.286
       0.5143  0.2057

3           3      15   18
        5.143  12.857
       0.8929  0.3571

4           4      16   20
        5.714  14.286
       0.5143  0.2057

5           7      13   20
        5.714  14.286
       0.2893  0.1157

All        28      70   98

Cell Contents:      Count
                    Expected count
                    Contribution to Chi-square


Pearson Chi-Square = 7.595, DF = 4, P-Value = 0.108

The second number in each of the ten cells is the expected count. All of them exceed 5, so the chi-squared statistic 7.595 can be properly compared with a chi-squared distribution with 4 degrees of freedom. The P-value 0.018 indicates that there are no differences among groups.

If the P-value were less than 0.05, then some ad hoc tests would be appropriate to see which groups had significant different probabilities of 'Yes' than others. A quick way to look for interesting results is to look at the third entry in each cell: 'Contributions to chi-squared'. If the overall P-value finds significant differences, then the 'Contributions' larger than 4 might point the way to them.

Also, you mentioned the Marascuillo procedure for formal ad hoc tests.

$\endgroup$
3
  • $\begingroup$ Thank you! 1) Can you please explain the 'contributions to chi-squared'? 2) I'm cant be sure that the expected distribution always exceeds 5. This is the main reason I prefer to use fisher also for mx2 tables. Are you familiar with something similar to marascuillo for fisher? $\endgroup$
    – niceguest
    Commented Aug 18, 2019 at 5:32
  • $\begingroup$ 3) Do you have something to say about testing the proportion against the expected proportion? (My main concern here is that the expected is the relative frequency of the group). It can be done either by the proportion test that I mentioned or by binomial proportion. $\endgroup$
    – niceguest
    Commented Aug 18, 2019 at 5:39
  • $\begingroup$ The chi-squared statistic above is the sum of ten quantities. Each of them is called a 'contribution' to chi-squared. I'm not sure what kind of test you contemplate in your second comment. If you have a specific proposal, please show it as an edit to your question, leave a Comment, and I (or someone else) will take a look. $\endgroup$
    – BruceET
    Commented Aug 19, 2019 at 0:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.