Null Hypothesis when sample is already true

Question

As a student I always learned a "rule" to work through my statistics problems: If the sample is true in my null hypothesis, then I'm doing things wrong and it should be the other way around. It worked most of the time in my exams.

Is there any reason or theory to backup this idea?

Example:

X is the mean of the sample. Sample returns X=24

The following would be "wrong" according to my rule, because the null hypothesis is already true in the sample:

Ho: X ≥ 22
Ha: X < 22

Then I would quickly turn things around to:

Ho: X ≤ 22
Ha: X > 22

Any idea why this works and what's the theory behind it? I used it in my exams and always worked.

Answer 1

There's no supporting theory, because this doesn't work except by chance alone.

Your choice of null and alternative hypotheses depend only on what you're trying to prove and not the sample value. Suppose you have some new drug that's supposed to lower cholesterol, and you want to test if it works. The null hypotheses is that the drug increases cholesterol or has no effect, while the alternative is that the drug actually does lower cholesterol. To claim that the drug does lower cholesterol, you need a significant amount of data to support that, typically enough to have an alpha of 0.05 - only 1 time in 20 will you say the drug has the intended effect when it actually does not.

If you flip your hypotheses arbitrarily, you're implicitly assuming that the drug works as intended, and requiring significant evidence to show that it does not. But that's not how proof works - a new claim requires positive evidence, you don't just assume it's true and then look for evidence it's false (which you could always fail to find just by collecting little evidence). Failing to show that the drug doesn't work as intended might just be a failure of your sample size, but is not alone evidence that it does work as expected. If you collect only 3 patients' worth of data, you won't have enough evidence to reject the possibility that the drug works as intended, but that's not meaningful at all. You should view that data as showing that you don't have evidence to conclude that the drug works, not that you've failed to rule out the drug's efficacy.

As you can see, none of this line of reasoning requires you to know anything at all about the cholesterol values actually observed in the population. How you test whether the drug works should not depend on whether the drug actually does work or not. The only reason this worked in your exams is because you got lucky defining the null and alternative hypotheses as your instructor intended (you had a 50/50 chance each time). The sample value has absolutely no role in defining the directionality of the hypothesis test.

Answer 2

You decide on your null and alternative hypothesis before you analyze the data (preferably before you even collect the data), so you have made a mistake in peeking.

By flipping the inequality to point away from the observed value, you assure yourself of not being able to reject (perhaps with some exotic exceptions, but this definitely applies to a t-test). That is, if you observe that $\bar{x} = 1$ and you try to show $H_a: \mu<0$, you will fail to reject $H_0: \mu\ge 0$ e(unless you have some silly $\alpha$ like $0.9$ instead of that usual suspects like $0.05$, $0.01$, and $0.1$).

But you should not be doing that, because you have not seen the calculation results when you decide on your null and alternative hypotheses.