# Why the null hypothesis should always be written as an equality

It is often stated that the null hypothesis should be written as an equality (see for example here), like for example $\mu_A-\mu_B=0$, while the alternative uses an inequality (for example, $\mu_B\gt\mu_A$). I think that this kind of null hypothesis is called "simple". My question is, why should we always prefer simple null hypotheses?

Suppose for example that I have an existing machine learning code A, and I develop a new code B, based on a different paradigm (maybe A is a neural network, while B is a random forest). I test A and B on some data. I can define an accuracy metric for the codes, thus for each test I can say whether A or B was more accurate. It seems extremely unlikely to expect that A and B could have the same average accuracy on the population from which the data are drawn. After all, they're completely different algorithms. Thus, a NH such as $H_0:p_A=p_B$ seems quite "unnatural" to me. Rather, since I want to know if new code is more accurate than the old one, I would try to falsify the NH that B is no more accurate than A, i.e., $H_0: p_B\le p_A$. Thus, the alternative would be $H_a:p_B\gt p_A$. Why is this kind of null not appropriate for testing?

I guess the issue is that with the "compound" null (is this the correct term?) I cannot easily find a distribution of a test statistics, and thus I may not be able to perform NHST. For example, in the two sample t-test, the null that the mean of the two samples is the same allows me to derive easily the distribution of the test statistics (mean is zero, and standard deviation is derived from sample standard deviation and number of samples). If instead I assume that one mean is larger than the other, then I don't know the parameters of the test statistics distribution. Is this correct?

• I'd say that not everyone agrees with the premise of the.title. Some people certainly take that point of view in respect of the usual tests, but some don't. Jun 7, 2016 at 3:02

There is nothing wrong with your proposed test. It is possible to derive the sampling distribution of the null with a compound null. What we do, in essence, is use the sampling distribution of the simple null, and if the truth were that $p_B\ll p_A$, then the rate of type I errors would be less than your stated alpha.

What you are getting at is called a non-inferiority test. Here is a non-gated and practitioner-friendly paper:

It may also help to read some of the existing threads on CV that are related to this topic. I have provided somewhat related answers:

If $$H_0: p_A-p_B=0$$ is not rejected with some significance level, then it doesn't imply that $$p_A$$ is exactly $$p_B$$ (this is very basic). It means that $$0$$ is in an interval around your observed test statistic. The two algorithms can be different but they can return values of some chosen loss function that are close from each other. This is reasonable. If you suspect that the algorithm B is at least as good as A's, you can test $$H_0:p_A=p_b$$ against $$H_1: p_A>p_B$$. This is what most of staticians would do....