Justification of one-tailed hypothesis testing

Question

I understand two-tailed hypothesis testing. You have $H_0 : \theta = \theta_0$ (vs. $H_1 = \neg H_0 : \theta \ne \theta_0$). The p-value is the probability that $\theta$ generates data at least as extreme as what was observed.

I don't understand one-tailed hypothesis testing. Here, $H_0 : \theta\le\theta_0$ (vs. $H_1 = \neg H_0 : \theta > \theta_0$). The definition of p-value shouldn't have changed from above: it should still be the probability that $\theta$ generates data at least as extreme as what was observed. But we don't know $\theta$, only that it's upper-bounded by $\theta_0$.

So instead, I see texts telling us to assume that $\theta = \theta_0$ (not $\theta \le \theta_0$ as per $H_0$) and calculate the probability that this generates data at least as extreme as what was observed, but only on one end. This seems to have nothing to do with the hypotheses, technically.

Now, I understand that this is frequentist hypothesis testing, and that frequentists place no priors on their $\theta$s. But shouldn't that just mean the hypotheses are then impossible to accept or reject, rather than shoehorning the above calculation into the picture?

Answer 1

That's a thoughtful question. Many texts (perhaps for pedagogical reasons) paper over this issue. What's really going on is that $H_0$ is a composite "hypothesis" in your one-sided situation: it's actually a set of hypotheses, not a single one. It is necessary that for every possible hypothesis in $H_0$, the chance of the test statistic falling in the critical region must be less than or equal to the test size. Moreover, if the test is actually to achieve its nominal size (which is a good thing for achieving high power), then the supremum of these chances (taken over all the null hypotheses) should equal the nominal size. In practice, for simple one-parameter tests of location involving certain "nice" families of distributions, this supremum is attained for the hypothesis with parameter $\theta_0$. Thus, as a practical matter, all computation focuses on this one distribution. But we mustn't forget about the rest of the set $H_0$: that is a crucial distinction between two-sided and one-sided tests (and between "simple" and "composite" tests in general).

This subtly influences the interpretation of results of one-sided tests. When the null is rejected, we can say the evidence points against the true state of nature being any of the distributions in $H_0$. When the null is not rejected, we can only say there exists a distribution in $H_0$ which is "consistent" with the observed data. We are not saying that all distributions in $H_0$ are consistent with the data: far from it! Many of them may yield extremely low likelihoods.

Answer 2

I see the p-value as the maximum probability of a type I error. If $\theta \ll \theta_0$, the probability of a type I error rate may be effectively zero, but so be it. When looking at the test from a minimax perspective, an adversary would never draw from deep in the 'interior' of the null hypothesis anyway, and the power should not be affected. For simple situations (the t-test, for example) it is possible to construct a test with a guaranteed maximum type I rate, allowing such one sided null hypotheses.

Answer 3

You would use a one-sided hypothesis test if only results in one direction are supportive of the conclusion you are trying to reach.

Think of this in terms of the question you are asking. Suppose, for example, you want to see whether obesity leads to increased risk of heart attack. You gather your data, which might consist of 10 obese people and 10 non-obese people. Now let's say that, due to unrecorded confounding factors, poor experimental design, or just plain bad luck, you observe that only 2 of the 10 obese people have heart attacks, compared to 8 of the non-obese people.

Now if you were to conduct a 2-sided hypothesis test on this data, you would conclude that there was a statistically significant association (p ~ 0.02)between obesity and heart attack risk. However, the association would be in the opposite direction to that which you were actually expecting to see, hence the test result would be misleading.

(In real life, an experiment that produced such a counterintuitive result could lead to further questions that are interesting in themselves: for example, the data collection process might need to be improved, or there might be previously-unknown risk factors at work, or maybe conventional wisdom is simply mistaken. But these issues aren't really related to the narrow question of what sort of hypothesis test to use.)