I have performed multiple comparisons using Mann-Whitney U tests, and want to correct the p-values to know which results are worth reporting. The structure of the data is as follows :
2 experiments, in which X different indicators were measured over time (Y time steps) in Z individuals.
The goal is to know if individuals from the 2 experiments differ in their indicators. As measures in each group need to be independent for Mann-Whitney, a test is run per time step. That's why I wonder if the correction should take into account that some tests are linked (different time steps of a same measure), and if yes, how can I do that?
I intend to plot the data when the tests indicate something significant, to check the effect is really there, and will probably only make strong conclusions if several time steps of a same indicator are detected as different between the groups.
I'd also like to know if the preliminary steps I performed are valid :
- The indicators measured sometimes fall below a limit of detection (LOD), so these will all be treated as ties by Mann-Whitney. So I excluded a priori comparisons for which there was less than 5 measures above limit of detection in each group (if one group had only measures at LOD and the other many measures above LOD, I think it's informative enough to run a test).
- Among the remaining comparisons, I test for equal variance using a Levene's test. I exclude comparisons with unequal variance (threshold of 5%, I don't correct p-values at this step). Here, cases where one group has only measures at LOD and the other shows variations end up being excluded...
- I run the Mann-Whitney tests on the remaining comparisons.