Multiple comparisons correction with independent groups A question: I have a study with 3 experiments, each involving a between-subjects comparison of a treatment vs. control group. The groups in each experiment are independent samples and no groups are repeated across experiments.  A t-test is run for each experiment.
A reviewer is asking for correction for multiple comparisons. It seems odd to perform such a correction across experiments, especially since we only ran one t-test per experiment.  However, in all 3 experiments we are interested in whether there is an effect of the treatment (albeit under different conditions in each case).  Is Bonferroni the appropriate method here?  What other methods might be preferable?  Are there citeable works on whether it is appropriate to do such corrections?
Note: a somewhat similar question was asked here: correcting for multiple comparisons with independent groups
Thanks for any advice!
 A: A Bonferroni correction is conservative, but always an option, because the Bonferroni inequality requires no assumptions about the relationships among the events. It exerts the strongest control over the simultaneous error probability, $\Pr(\text{at least one type I error})$, unconditionally, and may be used also for constructing (conservatively) a set of confidence intervals whose simultaneous confidence level exceeds the specified one. For just 3 tests, it usually isn’t all that conservative, and so it has much to recommend it when this strong type of control over the error rate is desired.
You could also use a slightly less conservative adjustment and run each test at an individual significance level of $1-(1-\alpha)^{1/3}$ (which is about 0.017 with $\alpha=0.05$). This is based on setting the joint probability of 3 independent events to the desired value, and it is justified because you have independence here. Again, this would control the overall error probability at the simultaneous CI level.
Some other simultaneous-testing procedures gain power by avoiding testing certain comparisons at all when other comparisons are already deemed non-significant. There are various levels of control over the error rate with such procedures. Methods that control the false-discovery rate are at the weakest level of control (other than no correction at all). It is unclear how one might apply such methods to the situation described in the OP. But I suppose one possibility is to fit a model with a block effect for the three experiments, and a treatment factor that has 4 (or 6?) levels: control, treat1--treat3 (or perhaps 3 levels of control?). A Dunnett-style test could be used to test the comparisons of interest. The Dunnett method has the same strong control over error rate as the Bonferroni, and be slightly more powerful than Bonferroni -- I suspect not that much. An FDR method could be applied using something like the R function p.adjust() with the unadjusted $P$ values.
A good reference for all the issues under discussion is Chapter 5 of Oehlert's experimental design text, which is openly available by the Creative Commons license: http://users.stat.umn.edu/~gary/book/fcdae.pdf 
A: Dunnetts multicomparrison test is used for multiple comparisons when there is a control
For reference:
http://www.statisticshowto.com/dunnetts-test/
