I'm trying to determine what constitutes a family of hypotheses when measuring multiple response variables. In particular I have an A/B type test where I'm comparing several marketing campaigns against a baseline campaign.
For each campaign I want to measure increase in revenue, clicks, and purchases. What I'm not sure of is how many families of hypotheses exist in this situation? Is it more correct to group tests together based on the response variable of interest, or should all the tests be considered one big family?
Concretely, assume I have 4 campaigns, and I want to compare each campaign against the baseline.
Does this mean I have three families of hypothesis, one each for revenue, clicks, and purchases, and each family contains 4 hypothesis? Or is it better to say I have one family of hypothesis, containing 12 individual hypotheses (4 comparisons over three variables)?