Do null and alternative hypotheses have to be exhaustive or not?

I saw a lot of times claims that they have to be exhaustive (the examples in such books were always set in such way, that they were indeed), on the other hand I also saw a lot of times books stating they should be exclusive (for example $\mathrm{H}_{0}$ as $\mu_1=\mu_2$ and $\mathrm{H}_{1}$ as $\mu_1>\mu_2$) without clarifying the exhaustive issue. Only before typing in this question I found somewhat stronger statement on the Wikipedia page -- "The alternative need not be the logical negation of the null hypothesis".

Could someone more experienced explain which is true, and I would be grateful for shedding some light on the (historical?) reasons for such difference (the books were written by statisticians after all, i.e. scientists, not philosophers).

On principle, there is no reason for hypotheses to be exhaustive. If the test is about a parameter $\theta$ with $H_0$ being the restriction $\theta\in\Theta_0$, the alternative $H_a$ can be of any form $\theta\in\Theta_a$ as long as $$\Theta_0\cap\Theta_a=\emptyset.$$

An example as to why exhaustivity does not make much sense is when comparing two families of models, $H_0:\ x\sim f_0(x|\theta_0)$ versus $H_a:\ x\sim f_1(x|\theta_1)$. In such a case, exhaustivity is impossible, as the alternative would then have to cover all possible probability models.

• Thank you, do you know by any chance why it is so common to see this requirement of being exhaustive? Apart from simple misunderstanding, because this would be one of most common misunderstandings :-). – greenoldman Nov 26 '11 at 12:03
• I do not understand the example. When you are comparing two families of models $H_0$ and $H_a$ between them do appear to exhaust every possible model in the family. If you allow the null and alternative not to cover every such model, you complicate the process of evaluating the decision-theoretic risk of the test (both in theory and in practice). – whuber Nov 27 '11 at 16:47
• @whuber: you misread my example. As written above, the alternative $H_a$ is made of a well-defined family of models,where $\theta_1$ ranges the whole set of possible values, rather than being made of all possible probability models. This is therefore not exhaustive. This is a criticism raised against the Bayesian approach to testing, see for instance the philosopher of science, Deborah Mayo, in Error and Inference – Xi'an Nov 28 '11 at 7:15
• I think I'm reading your example correctly, Xi'an, but clearly I'm struggling with what you mean by "exhaustive." Its use in your answer and comments appears to mean "includes all probability distributions," but in most hypothesis testing situations this is not relevant. In the present situation, "exhaustive" needs to mean "comprising all distributions included in the model" (such as all normal distributions for a normal-theory test). – whuber Nov 28 '11 at 15:36

The main reason you see the requirement that hypotheses be exhaustive is the problem of what happens if the true parameter value is in the region which is not covered by either the null or alternative hypothesis. Then, testing at the $\alpha %$ level of confidence becomes meaningless, or, perhaps worse, your test will be biased in favor of the null - e.g., a one-sided test of the form $\theta = 0$ vs. $\theta > 0$, when actually $\theta < 0$.

An example: a one-sided test for $\mu = 0$ vs $\mu > 0$ from a Normal distribution with known $\sigma = 1$ and true $\mu = -0.1$. With a sample size of 100, a 95% test would reject if $\bar{x} > 0.1645$, but 0.1645 is actually 2.645 standard deviations above the true mean, leading to an actual test level of about 99.6%.

Also, you rule out the possibility of being surprised, and learning something interesting.

However, one can also look at it as defining the parameter space to be a subset of what might typically be considered the parameter space, e.g., the mean of a Normal distribution is often considered to lie somewhere on the real line, but if we do a one-sided test, we are, in effect, defining the parameter space to be the part of the line covered by the null and alternative.

• Thank you, you made mistake in wording though, not exclusive but exhaustive (first line). – greenoldman Nov 27 '11 at 16:05
• Conceptually, a one sided test is really a test in the form $H_0: \theta \le 0$ vs. $H_A: \theta \gt 0$ rather than $H_0: \theta = 0$ vs. $H_A: \theta \gt 0$. In elementary expositions, especially those seen on the Web, this distinction is glossed over, but it is carefully and correctly handled in the statistical literature and good textbooks. Thus we are not restricting the parameter space. – whuber Nov 27 '11 at 16:46
• whuber - you're right about the one-sided test, of course. I was trying, albeit ineptly, to describe what might happen if the hypotheses were not in fact exhaustive, which in this case would come about if the null were $\theta = 0$. If we really want to stick with the point null and the one-sided alternative, and have exhaustive hypotheses, it seems to me we must redefine the parameter space as above. – jbowman Nov 28 '11 at 0:56
• Really @whuber? The null hypothesis in a one-sided test is an inequality that includes the untested tail? That makes so much more sense to me! But as you say, it was presented in my course as being a point equality. Thanks for the clarification. – James Jul 15 '18 at 22:14