5
$\begingroup$

I am working on some research where we are studying the effectiveness of two different types of brand new (i.e. never been studied before) treatments (call them high and low treatment levels) on death rates. There is a general belief among physicians involved in the study that the high level of treatment level will lead to lower death rates, but there is currently very little (or no) data to suggest this is the case -- it's just a hunch among the medical staff.

Before I carry out tests for differences in proportions between the treatment groups, I was wondering if it is more appropriate to use a two-sided test or a single-tailed test? This isn't a basic question of the difference between the a two-tailed and a single-tailed test, I fully understand that. This is more of a practical, "when the rubber meets the road," applied question about whether it's appropriate to use a one-sided or two-sided test when all we have to go on is a hunch that the high treatment will lead to significantly smaller number of deaths.

So, which do you think is a more appropriate alternative hypothesis for this test?

$$H_A: p_{high}-p_{low}<0$$

or

$$H_A: p_{high}-p_{low}\ne0$$

$\endgroup$
  • 2
    $\begingroup$ Why do you speak of "confidence intervals" in your title but don't mention them again in the question text? Can you edit your title to be more specifically related to what you are asking? $\endgroup$ – amoeba says Reinstate Monica Mar 30 '16 at 10:09
  • 1
    $\begingroup$ Great point. I meant 1-sided hypothesis test. I'm fixing that now. $\endgroup$ – StatsStudent Mar 30 '16 at 15:44
2
$\begingroup$

Hypothesis testing:

I refer to this answer: What follows if we fail to reject the null hypothesis?.

Hypothesis testing is about ''finding statistical evidence for your alternative hypothesis $H_A$'', i.e. whether the data you observe is (statistical) evidence that $H_A$ is true (see What follows if we fail to reject the null hypothesis? for more detail).

So if you want to ''show'' (with the data that you observe) that $p_{high} > p_{low}$ then your alternative hypothesis should be $H_A: p_{high} > p_{low}$ versus $H_0: p_{high} \le p_{low}$. (note that there is no ''$\hat{}"$ here because this is about the ''true'' values for $p_{high}$ and $p_{low}$.

This is a one-sided hypothesis, so this has nothing to do yet with the critical region being one-sided or two-sided. Note that we talk about a critical region, not about a confidence interval. (see below for explanation on confidence intervals).

Once you formulated your hypothesis, you can, using a test-statistic, define a critical region. If the data that you observe i.e. $\hat{p}_{high} - \hat{p}_{low}$ (here is where the "$\hat{}$" comes in, i.e. your data you observe) falls in the critical region then you reject $H_0$ and conclude that your data is evidence in favor of $H_A$. How will we now chose that critical region ? Well, we will, given a significance level $\alpha$ chose that critical region where we can most easily find evidence for $H_A$. In other words, we will chose the critical region with the highest power given $\alpha$. It happens that, for a (univariate) one-sided hypothesis (as you have) a one-sided critical region (i.e. an interval in the tail of the distribution of the test statistic) has more power.

Note that, if you want to show that the ''true'' $p_{high}$ is different from the ''true'' $p_{low}$ then you $H_A^{(1)}$ should be $H_A^{(1)}: p_{high} \ne p_{low}$ and the null should be $H_0^{(1)}: p_{high} = p_{low}$. In that case you're better of with a two-sided critical region.

But your alternative hypothesis is just what you want to show, so if you want to show that $p_{high} > p_{low}$ then this is your alternative hypothesis and your critical region is chosen to have the best power. If you want to show that $p_{high} = p_{low}$ then this should be your alternative (and then you're better of with a two-sided critical region). So it all depends on what the researcher wants to show: do you want to show that ''high treatment has more effect than low treatment'' or do you want to show that ''high and low treatment have a different effect'' ?

Confidence intervals

Confidence intervals are different from critical regions, see Why is there a need for a 'sampling distribution' to find confidence intervals? for explanation on confidence intervals.

Hypothesis testing use the observed data to find 'statistical evidence' for your hypothesis $H_A$ that is just a statement about the ''true parameters'' $p_{high}$ and $p_{low}$ (no $\hat{}$ because it is about the ''true'' values of these parameters.

A confidence interval is an interval (at a chosen confidence level) around the observed value $\hat{p}_{high} - \hat{p}_{low}$ (observed therefore the $\hat{}$) in which you are ''confident'' at a certain level that the interval will contain the ''true'' value of $p_{high}-p_{low}$.

Note that there is only one ''true'' value for $p_{high}$ and $p_{low}$ but we do not know these ''true'' values. On the other hand, each time we take a sample (i.e. each time we observe data) you will usually find another value for $\hat{p}_{high} - \hat{p}_{low}$. You want to use now one such an observation $\hat{p}_{high} - \hat{p}_{low}$ to make inferences about the (one) ''true'' value $p_{high}$ and $p_{low}$. There are two ways to make inferences: (1) make a statement about $p_{high}$ and $p_{low}$ and then ''check'' whether the observation confirms the statement (at a certain significance level), this is hypothesis testing and (2) use the observed data $\hat{p}_{high} - \hat{p}_{low}$ to construct an interval that contains (at a certain confidence level) the true value $p_{high}-p_{low}$. This is a confidence interval.

Finally note that hypothesis tests can be used to construct confidence intervals and vice versa.

$\endgroup$
4
$\begingroup$

A one-sided confidence interval (CI)/test is as good as a two-sided CI/test: it all depends on your assumptions and goals. Given what you are telling us, ie that the prior knowledge is very limited ('hunch'), using a one-sided approach is almost groundless, and you risk being not conservative enough. I would thus recommend to use a two-sided CI/test.

Choosing a shortcut such as a one-sided CI/test in this phase is likely going to backlash when the final report is submitted for dissemination/publication.

References which are also pertinent to your issue are those on inferiority vs equivalence trials:

http://www.ncbi.nlm.nih.gov/pubmed/26604186

http://www.ncbi.nlm.nih.gov/pubmed/24137721

http://www.ncbi.nlm.nih.gov/pubmed/22145119

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.