I have a study design and have been asked to test four hypotheses. I have a fairly good understanding of multiple comparisons when it comes to the same hypothesis i.e. if I have 3 levels then I need to correct for 3 comparisons, but am unsure whether I need to perform an extra step as I have multiple hypotheses tied around the same dataset.
To summarise what each hypothesis is asking.
H1 = is assessing how often a certain response is made following an event (4 groups)
H2 = is assessing whether the result following the event is the same of different between the four aformentioned groups.
H3 = assesses across all of those groups whether the time taken before event occurred is statistically different from another set of data.
H4 = assesses whether result following the event is statistically different from zero. One of the groups is omitted during this analysis because the data is appropriate for H2 but not for H4.
So all my hypotheses draw from the same population, but H1 draws from it in a completely different manner from any other, as does H3. H4 is using a subset of the data from H2 but is asking a different global question about the event whilst not caring about groups.
To make things more interesting, for H2, H3 and H4, each analysis was carried out assessing the result following the event after ten business days and also after 126 business days.
All my statistics have been corrected in a reasonable fashion within each hypothesis. However when looking at them together I am wondering whether it is sufficient that they are asking different questions or is the fact that they use the same underlying data an issue that requires correction?