I have a data set with about 6,000 Fishers exact tests. If I want to do a multiple testing correction to control the FWER what are the advantages of one of these methods vs the other

calculating a q-value for each test Cite

Using a Benjamini-Hochberg control of the FDR

Hochberg's step up method or Holm step down method Cite

Sorry if this is a stupid question, I'm a not a classically trained statistician

  • $\begingroup$ This is where the 'art' in statistics comes in. It depends on what you want your analysis to be like. Do you want to be ultra conservative? Do you want to control False Discovery rate or Family wise error rate? Do you want to control it in a strong sense or weak? I suggest reading up on the literature. Sometimes it might not matter. See what bonferroni correction (very naive, easy, and SUPER conservative) gives you and compare to something like FDR (much less conservative). $\endgroup$ – bdeonovic Oct 9 '13 at 19:54
  • $\begingroup$ Also, if you are using R, just want you to be aware that there is a great built in function called p.adjust which will adjust your p-values and has all of the methods you mentioned (and more) available. $\endgroup$ – bdeonovic Oct 9 '13 at 19:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.