1
$\begingroup$

I have dataset which looks like below -

Categories Sample 1 Sample 2 .... Sample 9 Reference Proportion
A 89 91 85 0.89
B 0 1 3 0.01
C 2 1 1 0.02
... 6 8 7 0.06
I 1 2 1 0.005

I want to test relative composition of the categories with respect to the "reference proportion". I am thinking of combining the count data from all the samples (Samples 1-9) , and then perform a chi square goodness of fit test against the reference proportion. By combining all the samples I make sure that all the categories have at least 5 values as required by the chi square test. But as chi square test does not tell me which categor(y/ies) were different from the reference, I am also thinking of doing a proportion test for each of the categories. Since I can get the proportion values for each sample and each category, I think I can perform a test for the proportion. But I am not sure which test of proportion to perform, since my number of samples are low (9)? Can you tell which should be ideal test to perform for each of the categories ? Also is it recommended to do both the chi square and individual proportion test? Since one gives an global overview of how things match and the other tells which is the deviant category which causing the trouble if any.

$\endgroup$

1 Answer 1

3
$\begingroup$

The Cochran's Rule that is often simplified to a "minimum of 5 expected counts per cell" is originally meant for contingency tables with degrees of freedom > 1 (see P. Kroonenberg & A. Verbeek, 2018). So it's not that evident that you should merge the "sample" categories together based on this rule, though merging them could prevent you from running underpowered tests – but maybe the question of merging or not these categories should be treated as a separate question, as you seem to focus this question on what kind of following-up tests you could use or not.

After conducting your goodness-of-fit test, two options appropriate for a chi-squared goodness-of-fit test are mentioned by D. Sharpe in Chi-Square Test is Statistically Significant: Now What? (2015):

  1. calculating residuals,
  2. comparing observed and expected cells.

I won't rewrite here what Sharpe already wrote very clearly in his open-access paper (see pages 2 and 3 of the pdf), but be sure to apply corrections to any following-up analysis you conduct after a chi-squared test, to limit the multiple testing problem. (Sharpe mentions some possible corrections in the paper).


References:

P. M. Kroonenberg & Albert Verbeek (2018) "The Tale of Cochran's Rule: My Contingency Table has so Many Expected Values Smaller than 5, What Am I to Do?", The American Statistician, 72:2, 175-183, DOI: https://doi.org/10.1080/00031305.2017.1286260

Sharpe, Donald (2019) "Chi-Square Test is Statistically Significant: Now What?," Practical Assessment, Research, and Evaluation: Vol. 20, Article 8. DOI: https://doi.org/10.7275/tbfa-x148

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.