-1
$\begingroup$

I am writing a meta-analysis on the complication and reoperation rates after 5 treatment modalities of distal radius fractures. Currently I am doing a post hoc analysis on the differences of 7 types of complications between the 5 treatment modalities seperately(4 comparisons, 4 of the treatments compared to the gold standard). I want to use the conservative Bonferroni correction for the alpha and read the other articles on multiple comparisons and the bonferroni correction. I am just not sure what number to divide the alpha by. Should I divide the alpha(which I set at 0.05) by 4 because of the 4 comparisons or not?enter image description here

$\endgroup$
1
$\begingroup$

Divide your alpha by the total number of hypothesis checks you are about to do in experiment.

$\endgroup$
  • $\begingroup$ For each complication type I had 4 hypotheses. Should I divide my alpha by 4 or by 28(4*the 7 types of complications)? $\endgroup$ – Rotterdam Jul 4 '17 at 13:29
  • 1
    $\begingroup$ If you want to state that your entire experiment (paper, study, or whatever) is significant at 1 - Alpha_global, then divide by 28 (all the pyhothesis included in your experiment). On the other hand if you want to state that complication A's family of tests is significant at 1 - Alpha_global, then divide by 4. However the later case implies that you do not bear a responsibility of your whole study being significant at the level you used. $\endgroup$ – Alexey says Reinstate Monica Jul 4 '17 at 13:38
  • $\begingroup$ The primary outcome measures are the complication rate and reoperation rate. 4 comparisons were done on both of them too. The types of complications are a secondary outcome measure. If I want to uphold to your first statement: should I actually divide my alpha by 4(complication rate)*4(reoperation rate)*28(4*7type complications)=448? $\endgroup$ – Rotterdam Jul 4 '17 at 13:45
  • 1
    $\begingroup$ Well, just compute (or estimate) how many Null Hypotheses are there that you are going to try to reject, using t-test or whatever method you prefer. If that number is 448 (the number of p-values you will compare to alpha), then do it. $\endgroup$ – Alexey says Reinstate Monica Jul 4 '17 at 14:03
  • 1
    $\begingroup$ Bonferonni method is designed for the most conservative case of independent hypotheses. It guarantees that your family-wise p-value (type I error chance in a whole study) <= global alpha in any hypothesis set, dependent or not. If you think your hypotheses maybe not independent (secondary will depend of primary) then Bonferroni's is too strict, but it still guarantees the result. I do not know in detail methods of alpha adjustment in case of certain hypothesis dependence. $\endgroup$ – Alexey says Reinstate Monica Jul 4 '17 at 14:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.