I already looked around quite a bit but didn't find answers (I understood), so I apologise if this question should be a duplicate.

I'm analysing a small cohort (n=15) of people with some observed difference/outcome X, which is either positive or negative. I want to check whether the number of positive X is different between one or many of three binary categories: Older than Y?, Born in Country Z?, and Previous Children?.

My naive approach was to construct three 2x2 contingency tables, do a Fisher's Exact test on each, and apply Bonferroni correction by multiplying the p-values times 3 (since I tested X vs Age, X vs Country, X vs Children).

So my question, as I'm not super statistically fluid: Is this correct? Are there better ways which also might take into account that the variables might be dependent on another (e.g. age and previous children)? I can't really combine the three variables into one since my sample is so small.


1 Answer 1


Because of the discreteness of the Fisher exact test, especially for such a small sample size, a simple Bonferroni correction will be too conservative. While accounting for dependence structure will reduce this conservativeness a little, it is hard to implement in your case. In any event, you can get much more power by applying the discrete Bonferroni correction in your example, so that is my recommendation.


Westfall, P.H. and Wolfinger, R.D.(1997). Multiple Tests with Discrete Distributions, The American Statistician 51, 3–8.

Also, in Chapter 17 of

Westfall,P., Tobias, R. and Wolfinger, R. (2011), Multiple Comparisons and Multiple Tests Using SAS (2nd Ed.), SAS® Press, Cary, NC

there are several worked out examples.

  • $\begingroup$ Thank's a lot! I will have a look at your suggestions and forward them to the coauthors. Will mark as solved afterwards :) $\endgroup$ Oct 29, 2020 at 12:35
  • $\begingroup$ Great! See also: Westfall, P.H. (2011). Improving Power by Dichotomizing (Even Under Normality), Statistics in Biopharmaceutical Research 3, 353–362. $\endgroup$ Oct 29, 2020 at 14:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.