I am interested in working out the correct way to correct for multiple testing in multilevel models for longitudinal data, where I am investing a potential interaction between two predictors.
My data consists of 7 outcome variables, 2 predictor variables with a interaction, and 5 covariates. For the model building strategy I am using a set up approach.
Model 1: Null model
Model 2: Time is introduced
Model 3: Covariates are introduced
Model 4: Predictors are introduced
Model 5: Interaction between Predictors are introduced.
This strategy is then repeated for each of the outcome variables.
Currently I am using bonferroni correction to account for multiple testing, .05/35 = .001, were 35 is 7 outcome variables multiplied by 5 models. Is this the correct way to perform the calculation for the bonferroni correction? Alternatively would you divide .05 by the number of final models in the analysis (7) or the number of parameters in the models been analyzed?
The following paper may be of interest to anyone coming across this post.
Evaluation of multi-outcome longitudinal studies
Evaluation of intervention effects on multiple outcomes is a common scenario in clinical studies. In longitudinal studies, such evaluation is a challenge if one wishes to adequately capture simultaneous data behavior. In this situation, a common approach is to analyze each outcome separately. As a result, multiple statistical statements describing the intervention effect need to be reported and an adjustment for multiple testing is necessary. This is typically done by means of the Bonferroni procedure, which does not take into account the correlation between outcomes, thus resulting in overly conservative conclusions. We propose an alternative approach for multiplicity adjustment that incorporates dependence between outcomes, resulting in an appreciably less conservative evaluation. The ability of the proposed method to control the familywise error rate is evaluated in a simulation study, and the applicability of the method is demonstrated in two examples from the literature.