I have a question about adjusting pvalues to account for multiple testing, as I need to do this given a small sample size with a lot of tests.
I have read that the most common and conservative method of controlling for the family-wise error rate is to use the bonferroni method (alpha / k, where k is the number of tests). I can also do this doing p.adjust on R. However, I have a question about k. Should k in this case be the number of comparisons overall, or should it be adjusted by the type of test? In other words, I have 8 t-tests, 8 chi-squares, and 8 Mann-Whitney tests. Should I do alpha/8, for each, or alpha/24?
This may be a simple question, but it has really gotten me stuck! I know the bonferroni correction is typically done for ANOVAs and it's done within each test, not for all other tests, so it is somewhat confusing.
Thanks for your help!