3
$\begingroup$

Let's say you wish to perform a one-sided hypothesis test in which you want to reject $H_0: \Delta \leq 0$ in favor of $H_1: \Delta >0$. You select this hypothesis test specifically because you are more interested in establishing $\Delta>0$ than $\Delta \neq 0$.

So you conduct a one-sided test and get a $p$-value of 0.04. You might report that this is a significant difference.

In line with recommendations of the ASA (section 4 of https://www.tandfonline.com/doi/full/10.1080/00031305.2016.1154108), you wish to supplement your finding with a confidence interval. Question: is it appropriate to use a two-sided confidence interval? On the one hand, these are generally more informative than a one-sided confidence interval, which has an upper bound of only infinity.

However, if you report a 95% two-sided confidence interval for $\Delta$, this will contain 0, because the two-sided p-value of the data is 0.08. This might confuse a reader with two apparently contradictory findings or perhaps be statistically inappropriate/incompatible with the one-sided hypothesis testing.

This answer explains that a one-sided hypothesis test is dual with a one-sided confidence interval, which I understand. It implicitly advocates a one-sided confidence interval. One sided confidence interval for hypothesis testing

But this answer instead advocates for pairing $\alpha=0.05$ hypothesis testing with a 90% two-sided confidence interval: Matching Confidence limits with One-Sided Hypothesis tests

So what should one do? Give a one-sided 95% CI, a two-sided 90% CI, or a two-sided 95% CI with a note of caution?

Improve this question
$\endgroup$
4
  • $\begingroup$ The "matching" question link you provide is asking about using confidence intervals "to do" testing. That is not the reason why some people recommend using confidence intervals to supplement testing. My preference would be to do a one-sided CI, but it is not essential. $\endgroup$
    – Graham Bornholt
    Commented May 19 at 12:01
  • $\begingroup$ The inferential purpose of a CI is not the same as a test. I can imagine a situation where I use, say, a 99% two-sided CI irrespective of any alpha level used in a one or two-sided test. $\endgroup$
    – Graham Bornholt
    Commented May 19 at 12:08
  • $\begingroup$ Question needs clarification or reframing. Re: "can" -- sure, absolutely a single paper can do both since it demonstrably does happen, so perhaps not a particularly useful question to answer. Meanwhile "should" is a matter of opinion. Whose opinion counts? referees / editors for some journal? colleagues in some application area? Likely my opinion will differ from many of those; I'm more likely to talk about choices and consequences of those choices (relating to what properties you want your analyses to have) than offer some arbitrary list of thou shalt nots. $\endgroup$
    – Glen_b
    Commented May 20 at 5:46
  • $\begingroup$ A question about this or that property of a set of analyses is one a statistician might have something meaningful to respond with. My guess at opinions of readers (or editors, etc) of some journal in another area outside my own may be next to useless; doubly so if I don't even know which area that might be. $\endgroup$
    – Glen_b
    Commented May 20 at 5:50

1 Answer 1

Reset to default
2
$\begingroup$

I think this is a subject on which reasonable people can differ and, as long as you clearly state what you are doing, you are OK. That is, you aren't wrong.

Personally, I would use a one sided CI with a one sided p value, but I would do this very rarely. You say you are "more interested" in one direction than another. Well, that's not enough for me. I would only do one sided tests and CIs if I was exclusively interested in one direction. Why? Because doing this diminishes your chance of being surprised, and being surprised can be good. As Herman Friedman, my favorite professor in grad school used to say: "If you aren't surprised, you haven't learned anything."

However, once I had made a decision to use one sided hypothesis tests, I would use one sided CIs, because this lessens the chance that readers will be confused and that my results will get copied badly by other researchers. And it just takes one mistaken interpretation of a result to start a chain reaction, especially if the result gets misquoted in a prestigious journal.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.