When to apply pvalues correction? I have what I believe to be quite a beginner question. I'm testing the effect of a qualitative variable (2 groups) on 4 response variables. These 4 variables are correlated (see figure below), but I'm interested (for this question) by the effect of the qualitative variable, which groups are colored in blue and red (for instance, test if the distribution of my two groups is different according to Var1).
See the figure below:

I know that pvalues correction for multiple tests must be applied when testing the difference of many groups on one response variable. However, in this case, I have only two groups, but they can be tested on different response variables. I wonder then if this situation falls into the application of p values correction. Moreover, the different responses being all correlated, I wonder if I really have to test the effect of the two groups on each of them?
 A: Some thoughts:

*

*Correction for multiple testing is a tricky issue, because it helps with the type I error probability at the price of decreasing the power, i.e., increasing the type II error probability. Personally I'd compute the p-values and avoid to interpret them in a binary "black/white"-manner. The error probabilities with which a test comes are valid without correction, so nothing stops you from calling a test with a p-value between 0.05 and 0.05/4 "significant at 5% level", however it is true that if you run four tests without correction, the probability of finding a false significance if nothing is going on is substantially higher than 5%, so significances may not be that meaningful. You can give those p-values below 0.05/4 or even 0.01/4 a stronger interpretation, like "there is strong evidence that something is going on here even after Bonferroni correction".


*The correlation between variables makes it hard to attribute separated effects to the different variables anyway, as they confound each other. As long as there's nothing known about causality, the effect of any variable tested in a separate manner could be a result of the indirect influence of other variables, or of interaction/joint effect.
