# Correction of p-values for multiple regression models with multiple comparisons

I was asked to do a correction of p-values for my analyses, but I'm not sure what p-values I am to correct, and presuming I'm using Bonferroni, what is the number that I am dividing the .05 level with. I have done three sets of multiple regression analyses, each with seven different time points for a total of 21 models. The method was backwards regression, independent variables remaining in the model at p < .05. The sets are comprised of the same group of independent variables entered, but the dependent variables differ (3 total x 7 time points). Due to the nature of backwards regression, obviously the final predictors in the model differ. Just to make the case clearer, by seven time points I mean the same variables have been measured at seven time points (i.e. A1, B1 to X1; A2, B2 to X2 etc).

So, what I really want to know is what is the correct way to go about this. Do I need to adjust the model ANOVA's p-values printed by SPSS, or the individual regression coefficients? Why this baffles me is the fact that in essence, a given time point's (potential) predictors are used for three different analyses. Then again, there are 7 actual regression analyses done for each set. This is of importance as some of the individual regression coefficients are close to .05 and if corrected, won't remain significant, whereas all the models even if corrected with what I think is the right level of alpha here, will remain significant.

• The issue is what are the hypotheses that you want to test simultaneously. The n you need would be the number of hypotheses. The p-value you are looking for is the probability of simultaneously rejecting all null hypotheses when the null hypotheses are true. Bonferroni is only one of several ways to approximate this p-value. In fact Bonferroni is very conservative in that it underestimates this p-value and is some cases by a large margin. You also could look at bootstrap and permutation methods (resampling approaches) as well as a method due to Tukey. Commented Jan 5, 2018 at 17:58

I can see three related issues:

A) What was your research question? Was there a main outcome and a clearly formulated hypothesis before you gathered data? The number of outcomes and predictors suggests not. If you are performing an exploratory analysis, say so in your report, refrain from making any strong conclusions, forget about p-value corrections, select the best outcome & model, and do a second experiment to replicate.