# Testing Hypothesis with different alternatives

I want to test whether $$\mu=\mu_0$$ where $$\mu_0$$ is some fixed number. Consider the following two different testings.

Hypothesis Testing 1: $$H_0:\mu=\mu_0,H_1:\mu<\mu_0$$

Hypothesis Testing 2: $$H_0:\mu=\mu_0,H_1:\mu>\mu_0$$

It is possible that testing 1 fails to reject $$H_0$$ and testing 2 rejects $$H_0$$. In this situation, I can't compare two testings as they are pear and apple.

Let's suppose you are not allowed to use $$H_1:\mu\neq\mu_0$$ here.

If I am in this situation, what should I do? Should I declare I can't reject $$H_0$$? Should I declare $$H_0$$ false by some probability? Which test should I trust?

Doing it right the first time is best. First, in practice, this should be an unlikely situation.

• Maybe you have re-engineered a pharmaceutical process hoping that the new process has a higher yield than the current one with $$\mu_0 = 100,$$ so you'd take data from runs of the new process, average them and test $$H_0: \mu= 100$$ vs. $$H_a: \mu > 100.$$
• Maybe your town has changed the widths of lanes and the sequencing of traffic lights on its main road hoping that the average late afternoon travel average travel time on the main stretch is reduced from the former $$\mu_0 = 20$$ min. Then you'd get travel times under the new configuration to test $$H_0: \mu = 20$$ vs $$H_a: \mu < 20.$$
• Maybe your old supplier whose product had 200mg of active ingredient per bottle has gone out of business and you are checking to see the amount of active ingredient in a former competitive supplier is the same as for the old supplier. Then you'd test $$H_0: \mu = 200$$ vs $$H_a: \mu \ne 200,$$ based on the average of $$n$$ randomly selected bottles from the prospective new supplier.

So ordinarily you would test one of the three kinds of tests and act according to the results of the test. One hopes you would have done a 'power and sample size' computation beforehand so you'd take a large enough sample $$n$$ in order to have a good chance (say 90%) of rejecting if there is a meaningful difference from $$\mu_0.$$ Then you would likely take the result of the one test as sufficiently good evidence to act upon.

But best-laid plans don't always work out. However, as a direct answer to your question, let's suppose you have taken data to test $$H_0: \mu = 100$$ vs. $$H_1: \mu < 100$$ at the 5% level, and cannot reject. Here are data simulated in R that would give such a result.

set.seed(806)
x = rnorm(10, 98, 15)
t.test(x, mu=100, alt="less")

One Sample t-test

data:  x
t = -0.69053, df = 9, p-value = 0.2536
alternative hypothesis: true mean is less than 100
95 percent confidence interval:
-Inf 104.6308
sample estimates:
mean of x
97.20135


The mean of my $$n=10$$ observations is $$\bar X = 97.2,$$ which is below the hypothetical mean $$\mu = 100,$$ but not enough smaller to be considered statistically significant. Maybe we put the wrong assumptions into our power computation so we didn't use a large enough $$n.$$ In this case, there's no use testing $$H_0: \mu = 100$$ vs. $$H_1: \mu > 100$$ because $$\bar X < 100$$ could never lead to rejection.

But what do we do if we guessed completely wrong and got data such as those in the simulation below?

set.seed(806)
x = rnorm(10, 110, 15)
t.test(x, mu=100, alt="less")\

One Sample t-test

data:  x
t = 2.2703, df = 9, p-value = 0.9753
alternative hypothesis: true mean is less than 100
95 percent confidence interval:
-Inf 116.6308
sample estimates:
mean of x
109.2014


Of course, we can't reject in favor of $$H_a: \mu < 100$$ based on a sample mean $$\bar X = 109.2.$$ Then we might be tempted to try testing $$H_0: \mu = 100$$ vs. $$H_1: \mu > 100.$$ [In R, the notation p.val gives just the P-value of the test, not the full printout.]

t.test(x, mu=100, alt="gr")$p.val [1] 0.02466914  So we could have rejected a test of $$H_0: \mu = 100$$ vs. $$H_1: \mu > 100$$ at the 5% level because the P-value $$0.025 < 0.05 = 5\%.$$ Doing multiple tests on the same data is always dangerous. If we try enough different things, we might accidentally get a rejection on one of our tries--just by chance. (The result would be a 'false discovery'.) Rejecting at the 2% level isn't a really strong result, but if is really important to resolve the true value of $$\mu,$$ then we might consider getting fresh data and doing the right test the second time. Or maybe making a two-sided 95% confidence interval to get a good guess at the actual value of $$\mu$$ in order to plan our course of action. Imagine testing $$\mu=0$$. You do your calculations and find that $$\bar{x}=99$$ and your z-statistic (or t-stat) is 123. I would have serious doubts about hypothesis 1 and very much believe hypothesis 2. • Sure. However, this could happen by chance. Say you can run experiment once. Commented Aug 7, 2020 at 1:10 • That’s a lot of bad luck if you get that result by chance. But everything always could have just happened by chance. The point of a p-value is to measure how surprising of a result you have if the null hypothesis is true. A very surprising result is evidence against the null hypothesis. – Dave Commented Aug 7, 2020 at 1:12 • Of course, one tries to do intelligent planning from the start. No guarantee against bad luck, but leads to surprisingly less 'bad luck'. Anyhow, if$T=123\$ really occurred, then I'd 'very much believe' hypothesis 2 (+1). Commented Aug 7, 2020 at 2:05

The research question is king. The job of the hypothesis test is to answer the research question. The job of the data (and statistics) is to help you perform the hypothesis test.

I think you are somewhat confused about how to set up your hypotheses. Your hypotheses should be formed from your research question and before you even look at your data! In particular, let's consider your statement

"Let's suppose you are not allowed to use $$H_1: \mu \neq \mu_0$$ here."

There is no such circumstance! You are always allowed to specify any valid alternative hypothesis you like, and this $$H_1$$ is a valid alternative hypothesis except in the pathological case where $$\mu$$ can only possibly take the single value $$\mu_0$$.

If you are struggling to work out which alternative hypothesis to use, it is easiest to write out what you are trying to test in words and then try to put it into algebra. Dave gives some examples of this in his answer, but the question will fall into one of three categories:

(A) You want to see if the mean is above some threshold or not. If the mean is $$\leq$$ the threshold you don't really care whether it is equal or lower. (Maybe you are seeing if a new, expensive drug is more effective than an existing cheap drug. If it isn't more effective then you don't care whether it is equally or less effective, you won't pursue it further because it is expensive.)

(B) You want to see if the mean is below some threshold or not. If the mean is $$\geq$$ the threshold you don't really care whether it is equal or higher. This is just the reverse of (A).

(C) You want to see if the mean is different from some value or not. (was Don Bradman a "100-runs-per-innings" batsman?)

All of these are legitimate research questions. A and B translate to one-sided hypothesis tests, C translates to a two sided test. However, we could have formulated research questions which were asking whether the two drugs had the same efficacy as each other (this is important for regulatory reasons in some cases), or whether Don Bradman's batting skill was over 100 runs per over. Those would have led to different alternative hypothesis.

Yes, it is possible that test 1 fails to reject $$H_0$$ and testing 2 rejects $$H_0$$. (Consider for example a t-test with level 1% on $$n = 9$$ datapoints where $$s^2 = 1$$, $$\overline{X} = \mu_0 + 3$$ ).

In such a case, you have to... wonder what is the relevant alternative, and this depends on what you want to test.

Keep in mind that one only rejects the null hypothesis in favor of the alternative. And one can only have a positive conclusion in case of a reject of $$H_0$$. Not rejecting $$H_0$$ does not allow you to accpet it.

So in the case you described, you cannot conclude that $$\mu < \mu_0$$ (since testing 1 didn't reject $$H_0$$) but you can conclude that $$\mu > \mu_0$$ (since testing 2 did reject $$H_0$$). If you just want to know if $$\mu = \mu_0$$ then use the alternative $$\mu \neq \mu_0$$.