In the context of a two-sided test, in his book Statistical Issues in Drug Development Stephen Senn writes chapter 13 (Determining sample size), p.201:
if we reject $H_{0}$, the hypothesis which we assert is $H_{1}$, which simply states that the treatment difference is not zero or, in other words, that there is a difference between the experimental treatment and placebo. This is not a very exciting conclusion but it happens to be the conclusion to which significance in a conventional hypothesis test leads [...] by observing the sign of the treatment difference, we are also justified in taking the further step of deciding whether the treatment is superior or inferior to placebo. A power calculation, however, merely takes a particular value, $\Delta$, within the range of possible values of given by $H_{1}$ and poses the question: ‘if this particular value happens to obtain, what is the probability of coming to the correct conclusion that there is a difference?’
If we wish to say something about the difference which obtains, then it is better to quote a so-called ‘point estimate’ of the true treatment effect, together with associated confidence limits.
From this, I understand that, even if two-sided test alternative hypothesis is non directional, one could determine the direction of the difference by looking at the sign of the estimate difference
But in the previous chapter 12 (One-sided and Two-sided Tests and other Issues to Do with Significance and P-values)p.185, about the statement "Unless we use one-sided tests when comparing a treatment with a placebo, we shall not be entitled to claim superiority of the treatment under any circumstances" he writes:
This argument is not reasonable on two counts (Hauschke and Steinijans, 1996) [...] Second, it is obvious that if we test $H_{1\ A}:\mu > 0$ against $H_{0}:\ \mu = 0$ at size $\alpha/2$ and also $H_{1\ B}:\mu < 0$ against $H_{0}:\ \mu = 0$ at size $\alpha/2$, then since it is impossible for both hypotheses to be rejected, the probability of rejecting one or the other given that $H_{0}$ is true is $\alpha/2 + \alpha/2 = \alpha$ . So, by carrying out this procedure we shall maintain the type I error rate at $\alpha$. Furthermore, we shall always reject the null hypothesis where the conventional two-sided test does so, and only reject the hypothesis when this does so. Hence these two one-sided tests are equivalent to the two-sided test. Therefore, carrying out a two-sided test and using the results to determine whether superiority, equivalence or failure to prove a difference applies is a perfectly acceptable procedure
From that, I understand that confirmatory directional tests (and hence, multiple comparisons) are needed (and possible) to conclude about the sign of the true difference
$\ $
So here are my questions:
- In the context of a significant difference with a two-sided test, is looking at the sign of the point estimate sufficient or not to induce the sign of the true difference?
- In the latter case, how does one run a confirmatory directional test in R?
Thanks,
- Statistical Issues in Drug Development, 2nd Edition. Stephen Senn © 2007 John Wiley & Sons, Ltd. ISBN: 978-0-470-01877-4
- Hauschke D, Steinijans VW (1996) Directional decision for a two-tailed alternative [letter;comment]. Journal of Biopharmaceutical Statistics 6: 211–218.
