When conducting pairwise comparisons, what restrictions are there on the alternative hypotheses? That is to say, is it okay if some tests are two tailed while others are one tailed?
$\begingroup$
$\endgroup$
5
-
2$\begingroup$ Your question may need to be more clearly focused -- what is "okay" depends on who is doing the okaying. Personally if you have a clear theoretical a priori basis for mixing them like that then there should be no inherent problem (though the outcomes of the pairwise tests could be expected to correspond even less well with an omnibus test than is usually the case). ... ctd $\endgroup$– Glen_bJan 3, 2019 at 5:05
-
2$\begingroup$ ctd ... However, I have more than once been publicly excoriated at great length by people from other areas of study who would not accept any suggestion of one-sided tests under any circumstances whatever. If you run into one like that (for example as a referee on a paper), nobody here could make guarantees that what is demonstrably fine algebraically (in the sense that suitable probability statements should continue to hold) would be remotely acceptable to them nor that you stood any chance of convincing them otherwise. $\endgroup$– Glen_bJan 3, 2019 at 5:05
-
2$\begingroup$ More reasonably, some people may hold doubt that your choices of which tests to do one-sided really were made before the fact, particularly if those choices made a difference in many of the cases. You may well need to publicly establish the hypotheses ("pre-register") before data collection to have a hope of convincing people in general (though you still would not hope to convince such people as I refer to above). $\endgroup$– Glen_bJan 3, 2019 at 5:11
-
2$\begingroup$ I agree with @Glen_b; I also would like to comment on the notion of "okay"ness in statistics more generally. People often ask statisticians if something is "okay". As Robert Abelson points out in his wonderful book Statistics as Principled Argument it's often not so much whether it is statistically okay as whether you can justify it. $\endgroup$– Peter FlomJan 3, 2019 at 9:50
-
$\begingroup$ @Glen_b I agree that at times there is a big difference between valid statistical practices and what reviewers will or won't allow. Your advice to justify different post hoc tests based on previous research is a strong and appreciated point. My suspicion is that it doesn't matter if the tests have different alternative hypotheses, but I would still like to find a reference showing why. $\endgroup$– ChernoffJan 3, 2019 at 23:26
Add a comment
|