# Do null and alternative hypotheses have to be exhaustive or not?

I saw a lot of times claims that they have to be exhaustive (the examples in such books were always set in such way, that they were indeed), on the other hand I also saw a lot of times books stating they should be exclusive (for example $\mathrm{H}_{0}$ as $\mu_1=\mu_2$ and $\mathrm{H}_{1}$ as $\mu_1>\mu_2$) without clarifying the exhaustive issue. Only before typing in this question I found somewhat stronger statement on the Wikipedia page -- "The alternative need not be the logical negation of the null hypothesis".

Could someone more experienced explain which is true, and I would be grateful for shedding some light on the (historical?) reasons for such difference (the books were written by statisticians after all, i.e. scientists, not philosophers).

On principle, there is no reason for hypotheses to be exhaustive. If the test is about a parameter $\theta$ with $H_0$ being the restriction $\theta\in\Theta_0$, the alternative $H_a$ can be of any form $\theta\in\Theta_a$ as long as $$\Theta_0\cap\Theta_a=\emptyset.$$

An example as to why exhaustivity does not make much sense is when comparing two families of models, $H_0:\ x\sim f_0(x|\theta_0)$ versus $H_a:\ x\sim f_1(x|\theta_1)$. In such a case, exhaustivity is impossible, as the alternative would then have to cover all possible probability models.

• Thank you, do you know by any chance why it is so common to see this requirement of being exhaustive? Apart from simple misunderstanding, because this would be one of most common misunderstandings :-). Nov 26, 2011 at 12:03
• I do not understand the example. When you are comparing two families of models $H_0$ and $H_a$ between them they do appear to exhaust every possible model in the family. If you allow the null and alternative not to cover every such model, you complicate the process of evaluating the decision-theoretic risk of the test (both in theory and in practice).
– whuber
Nov 27, 2011 at 16:47
• @whuber: you misread my example. As written above, the alternative $H_a$ is made of a well-defined family of models,where $\theta_1$ ranges the whole set of possible values, rather than being made of all possible probability models. This is therefore not exhaustive. This is a criticism raised against the Bayesian approach to testing, see for instance the philosopher of science, Deborah Mayo, in Error and Inference Nov 28, 2011 at 7:15
• I think I'm reading your example correctly, Xi'an, but clearly I'm struggling with what you mean by "exhaustive." Its use in your answer and comments appears to mean "includes all probability distributions," but in most hypothesis testing situations this is not relevant. In the present situation, "exhaustive" needs to mean "comprising all distributions included in the model" (such as all normal distributions for a normal-theory test).
– whuber
Nov 28, 2011 at 15:36
• This is not an answer to the question. Practical impossibility of types of exhaustive test does not at all imply that tests should not be exhaustive. It may explain why people do not deal with them, but that is something different. It would actually point to a weakness in the method of hypothesis testing. An analogy: a large percentage of people in a country make mistakes in filling out their tax form, because it is too difficult for them. That doesn't justify making mistakes when filing ones taxes, but should lead to the conclusion that the tax forms should be replaced with clearer ones. Sep 20, 2020 at 14:16

The main reason you see the requirement that hypotheses be exhaustive is the problem of what happens if the true parameter value is in the region which is not covered by either the null or alternative hypothesis. Then, testing at the $\alpha %$ level of confidence becomes meaningless, or, perhaps worse, your test will be biased in favor of the null - e.g., a one-sided test of the form $\theta = 0$ vs. $\theta > 0$, when actually $\theta < 0$.

An example: a one-sided test for $\mu = 0$ vs $\mu > 0$ from a Normal distribution with known $\sigma = 1$ and true $\mu = -0.1$. With a sample size of 100, a 95% test would reject if $\bar{x} > 0.1645$, but 0.1645 is actually 2.645 standard deviations above the true mean, leading to an actual test level of about 99.6%.

Also, you rule out the possibility of being surprised, and learning something interesting.

However, one can also look at it as defining the parameter space to be a subset of what might typically be considered the parameter space, e.g., the mean of a Normal distribution is often considered to lie somewhere on the real line, but if we do a one-sided test, we are, in effect, defining the parameter space to be the part of the line covered by the null and alternative.

• Thank you, you made mistake in wording though, not exclusive but exhaustive (first line). Nov 27, 2011 at 16:05
• Conceptually, a one sided test is really a test in the form $H_0: \theta \le 0$ vs. $H_A: \theta \gt 0$ rather than $H_0: \theta = 0$ vs. $H_A: \theta \gt 0$. In elementary expositions, especially those seen on the Web, this distinction is glossed over, but it is carefully and correctly handled in the statistical literature and good textbooks. Thus we are not restricting the parameter space.
– whuber
Nov 27, 2011 at 16:46
• whuber - you're right about the one-sided test, of course. I was trying, albeit ineptly, to describe what might happen if the hypotheses were not in fact exhaustive, which in this case would come about if the null were $\theta = 0$. If we really want to stick with the point null and the one-sided alternative, and have exhaustive hypotheses, it seems to me we must redefine the parameter space as above. Nov 28, 2011 at 0:56
• Really @whuber? The null hypothesis in a one-sided test is an inequality that includes the untested tail? That makes so much more sense to me! But as you say, it was presented in my course as being a point equality. Thanks for the clarification. Jul 15, 2018 at 22:14

Since a hypothesis test only has two options – reject the null or fail to reject the null – the alternative hypothesis simply stands in as a way to describe what rejecting the null looks like. Thus, it must be the compliment / negation of the null hypothesis for it to mean anything in regards to the outcome of a hypothesis test.

But as many have previously commented, partitioning the universe into two hypotheses makes for a very broad alternative hypothesis, so then we should say that given a set of assumptions that cover most possibilities, the alternative hypothesis should be the compliment of the null.

The alternative does not need to be exhaustive neither the null hypothesis must necessarily mean something different than what it states. For example, consider three independent observations $$X_i\sim N(\mu_i,1)$$, $$i=1,2,3$$, where $$(\mu_1,\mu_2,\mu_3)\in\mathbb R^3$$ and the problem of testing $$H_0:\mu_1=\mu_2=\mu_3$$ vs $$H_1:\mu_1\leqslant\mu_2\leqslant\mu_3$$ with at least one inequality being strict. This is a basic problem of order restricted inference, see e.g. Robertson, Wright and Dykstra (1988, Order Restricted Statistical Inference) or Silvapulle and Sen (2005, Constrained Statistical Inference: Inequality, Order, and Shape Restrictions). In this problem the null hypothesis does not mean anything else than that the three distributions coincide while the two hypotheses are not exhaustive.