Simple and paired t-tests and the control for multiple comparisons Let's say that I have a group of subjects that were measured in two conditions (A and B) and I want to check three hypotheses: i) do data from A and B differ from each other?, ii) do data from A differ from zero?, iii) do data from B differ from zero?.
So, my tests would be: a paired t-test for hypothesis i), and two simple t-tests for hypotheses ii) and iii).
I'm wondering whether I should or I shouldn't correct the alpha level of these three tests. I'm inclined to apply the correction for multiple comparisons to avoid type I errors, but at the same time I would also like to avoid type II errors.
 A: Independently of the specifics of the tests you are considering, a general solution is to perform a power analysis. Set your error level at .05/3 and see what power you would attain with the effect you can expect and the sample size you intend to use. If power is too low/risk of type II error is too high for your taste, you have two (and a half) options:


*

*Increase sample size

*Give up multiple comparisons correction/increase error level (there is nothing sacred about 5% and nothing special about the correction for multiple comparisons, you can always set a higher error level to increase power; of course, you also need to get away with it publication-wise if that's your objective)


The half option:


*

*Run another experiment (use measures with higher reliability, study a more homogeneous sub-population, make the experimental design within-subject, create a stronger effect by using another manipulation, etc.) or give up altogether (If there is no reasonable chance to identify a plausible effect, why spend time and money on it? Even if you do find a “significant” effect, it's likely to be an gross overestimation of the actual magnitude of the phenomenon.)



EDIT (following comment)
The important thing to understand is that power depends on alpha (the type I error rate), the sample size and the magnitude of the effect. If costs don't matter, you can always use an adjusted alpha and still attain the same level of power by increasing the sample size, thus making the problem disappear.
Assuming a fixed sample size, any decision regarding corrections for multiple comparisons will depend on what levels of type I and type II errors are acceptable. The trouble with any “standard practice” in this respect is that it wouldn't take the trade-off between the two into account. If the effect size is huge, a conservative type I error level will not make power unacceptably low. If the effect is tiny, sticking religiously to the 5% error level means giving type I errors enormous weight over type II errors.
It does not make sense not to factor this into the decision. Arguments about this problem (e.g. the call not to use corrected tests in epidemiology) rely on implicit assumptions about effect sizes and the relative costs of type I and type II errors (e.g. not identifying risk factors vs. falsely flagging something as risky).
A: When considering whether to correct for multiple comparisons I like to think about how my conclusions might change if I generated a bunch of completely random noise, gave it a plausible sounding name, and added it to my analysis.
Here are 2 extreme (and oversimplified) cases to use as example.
Case 1:  I am fishing for anything significant, I have thrown a bunch of variables in my model that I think may possibly have a relationship with my outcome and if any of them are significant then I will claim success.  
In this case if I start adding in random noise variables and don't adjust for multiple comparisons, then by chance alone I will find something significant if I add enough noise variables.  So I should adjust here.
Case 2:  I am very interested in the answer about A, but while doing the experiment I collect additional information as well (the added cost is minimal and could be useful for future research).
Here if a noise variable C is significant it does not (or at least should not) change my conclusions about A.  So here I would not adjust.
Most cases will probably fall between the above, but it is a useful thought process.  You can always compute both and look at the adjusted and unadjusted values.  If they agree then there is really no problem, if one set shows significance but the other does not, then you can still report both and let the reader decide (or go back and collect more data to better answer the question).
For your case I think I would start with an overall test that simultaneously tests A=B=0, if that is not significant then it is probably not worth looking at the individual tests.  If it is significant then you can look at the individual tests, this is called a protected test.
