Correlation analysis for a single sample: Correcting for multiple comparisons? I ran a study with a single group of participants and with 12 variables. I conducted 2 planned regression analyses (i.e., I hypothesized a specific predictor for 2 DVs prior to running the experiment), but I first conducted exploratory correlations between the 12 variables to evaluate potential covariates. Do I need to correct for multiple comparisons? Why or why not, and if so, is there a preferred method for this situation? Also, if I need to adjust, do I need to do so only for the set of exploratory correlations, or also for the planned regression analyses?
 A: It is worth to think back why one corrects for multiple comparisons. If you work in the Neyman-Pearson framework (which you probably do), you use inferential statistics (for example p-values) in order to reduce the number of times you will be wrong (in the long term). Notice how being wrong is not related to statistics per se, but rather to the theoretical inferences which you as a researcher attach to them. So, regarding the 2 planned tests you performed, the need for a correction depends on how they are related to the claims you want to make.
Scenario 1: you claim there is an association if either outcome variable ('DV') is associated with the predictor
In this case, you need to control for multiple comparisons, as otherwise the type I error rate increases from 5% to 9.75%.
Scenario 2: you claim there is an association if, and only if, both outcome variables ('DV') are associated with the predictor
In this case, you do not need to control for multiple comparisons, quite the opposite. You can increase the alpha level to 0.2236. Under the null, a p-value below this level in both tests only has a chance of 0.2236 * 0.2236 = 0.05.
Scenario 3: you wanto to make two independent theoretical claims, one for the association between predictor and variable 1, and another claim for predictor and variable 2.
In this case, you do not need to control for multiple comparisons as the theoretical inferences are independent.
For more on this, I can highly recommend this blog post by Daniel Lakens.
As to your exploratory work, given that it was not planned in advance I do not believe that p-values are interpretable in the Neyman-Pearson framework. I would report them uncorrected and clearly indicate that these are exploratory analyses warranting replication.
