Exploratory study with multiple testing

Question

I have conducted a study with 20 participants.

During the study I measured 9 different plasma hormones at 5 different times before and after an intervention.

As an exploratory study I would like to compare the hormone levels before and after the intervention to see if they differ. Even though the same hormone has been measured 5 times, I expect the values to be different at each time point because of cyclic variation, but the 5 time points before and after the intervention are measured at the same time in the cycle and should therefore be comparable minus the possible intervention effect (if that makes sense).

If I conduct paired samples T-tests i will end up with 45 tests. Should I worry about multiple testing and do some kind of post-hoc adjustment (e.g. Bonferroni) or is there a better way to go about this?

Alternatively, is it enough to state that this is an exploratory study and that the results have to be confirmed in other well-powered studies?

Thanks

Answer 1

In my opinion the best thing to do is to analyze the evolution of the plasma hormones over time and according to the presence or absence of intervention with a mixed effect model. Your dataset should have the following columns :

• One column "intervention" in your dataset containing the levels "before" and "after".
• One column "time point" containing the 5 levels being the time points in the cycle.
• Since you have repeated measures per individual, you should have one column containing the name of the individual, which will be a random factor.
• And then one column containing the value of the level of the hormone in the plasma. You would need one table per hormone, or you can have one "hormone" column and a bigger table, in case you want to test whether the levels of several hormones are correlated.

Then depending on the distribution of the hormone level in the plasma, I would say probably from the Poisson family, you could run a glmer (with the lme4 package for example). In R this would look roughly like this :

plasma hormone level~time point*intervention+(1|individual)

If the interaction is significant, it would tell you that the effect of the intervention differs according to the time point considered. If not, you expect at least a simple effect of the time point because of these cycles that you mention. Then you can take a decision from here. I would not run a post hoc adjusted for multiple comparisons, instead I would use the "effects" package to plot the effects of the best model with the confidence intervals around the estimates as advised here : https://psycnet.apa.org/record/2005-01817-003