Is it possible to accept the alternative hypothesis?

Question

I'm aware of several related questions here (e.g., Hypothesis testing terminology surrounding null, Is it possible to prove a null hypothesis?) but I don't know the definitive answer for my question below.

Suppose a hypothesis test where we want to test whether a coin is fair or not. We have two hypotheses:

$H_0: p(head)=0.5$

$H_1: p(head)\neq0.5$

Suppose we use 5% significance level, there are two possible cases:

1. When we obtain the data and find that the p-value is less than 0.05, we say "With significance level 5%, we reject $H_0$."
2. p-value is greater than 0.05, then we say "With 5% significance level, we cannot reject $H_0$."

My question is:

In case 1, is it correct to say "we accept $H_1$"?

Intuitively, and from what I have learned in the past, I feel that "accepting" anything as a result of hypothesis testing is always incorrect. On the other hand, in this case, since the union on $H_0$ of $H_1$ covers the whole "space", "reject $H_0$" and "accepting $H_1$" look exactly the same to me. On another thought, I can also think of the following idea, which says it is incorrect to say "we accept $H_1$":

We have an evidence strong enough to believe that $H_0$ is not true, but we may not have an evidence strong enough to believe that $H_1$ is true. Therefore, "rejecting $H_0$" does not automatically imply "accepting $H_1$"

So, what is the right answer?

Answer 1

IMO (as not-a-logician or formally trained statistician per se), one shouldn't take any of this language too seriously. Even rejecting a null when p < .001 doesn't make the null false without a doubt. What's the harm in "accepting" the alternative hypothesis in a similarly provisional sense then? It strikes me as a safer interpretation than "accepting the null" in the opposite scenario (i.e., a large, insignificant p), because the alternative hypothesis is so much less specific. E.g., given $\alpha=.05$, if p = .06, there's still a 94% chance that future studies would find an effect that's at least as different from the null*, so accepting the null isn't a smart bet even if one cannot reject the null. Conversely, if p = .04, one can reject the null, which I've always understood to imply favoring the alternative. Why not "accepting"? The only reason I can see is the fact that one could be wrong, but the same applies when rejecting.

The alternative isn't a particularly strong claim, because as you say, it covers the whole "space". To reject your null, one must find a reliable effect on either side of the null such that the confidence interval doesn't include the null. Given such a confidence interval (CI), the alternative hypothesis is true of it: all values within are unequal to the null. The alternative hypothesis is also true of values outside the CI but more different from the null than the most extremely different value within the CI (e.g., if $\rm CI_{95\%}=[.6,.8]$, it wouldn't even be a problem for the alternative hypothesis if $\mathbb P(\rm head)=.9$). If you can get a CI like that, then again, what's not to accept about it, let alone the alternative hypothesis?

There might be some argument of which I'm unaware, but I doubt I'd be persuaded. Pragmatically, it might be wise not to write that you're accepting the alternative if there are reviewers involved, because success with them (as with people in general) often depends on not defying expectations in unwelcome ways. There's not much at stake anyway if you're not taking "accept" or "reject" too strictly as the final truth of the matter. I think that's the more important mistake to avoid in any case.

It's also important to remember that the null can be useful even if it's probably untrue. In the first example I mentioned where p = .06, failing to reject the null isn't the same as betting that it's true, but it's basically the same as judging it scientifically useful. Rejecting it is basically the same as judging the alternative to be more useful. That seems close enough to "acceptance" to me, especially since it isn't much of a hypothesis to accept.

BTW, this is another argument for focusing on CIs: if you can reject the null using Neyman–Pearson-style reasoning, then it doesn't matter how much smaller than $\alpha$ the p is for the sake of rejecting the null. It may matter by Fisher's reasoning, but if you can reject the null at a level of $\alpha$ that works for you, then it might be more useful to carry that $\alpha$ forward in a CI instead of just rejecting the null more confidently than you need to (a sort of statistical "overkill"). If you have a comfortable error rate $\alpha$ in advance, try using that error rate to describe what you think the effect size could be within a $\rm CI_{(1-\alpha)}$. This is probably more useful than accepting a more vague alternative hypothesis for most purposes.

---

^{* Another important point about the interpretation of this example p value is that it represents this chance for the scenario in which it is given that the null is true. If the null is untrue as evidence would seem to suggest in this case (albeit not persuasively enough for conventional scientific standards), then that chance is even greater. In other words, even if the null is true (but one doesn't know this), it wouldn't be wise to bet so in this case, and the bet is even worse if it's untrue!}

Answer 2

Assuming that by throwing the coin several times you get the sequence (head, tail, head, head, head)

What you truly compute with hypothesis testing is actually ℙ[ obtaining (head, tail, head, head, head) | ℙ(head) = 0.5 ]

That is, you get an answer to the following question:

Assuming H0: ℙ(head) = 0.5, do I get the sequence (head, tail, head, head, head) at least 5% of the time?

So the question is formulated in such a way that you simply cannot get the answer as formulated in 1. Is ℙ(head) ≠ 0.5 true?

Both statements are not mutually exclusive. It is not because one proposition is proven wrong that another is necessarily true.

So in case 1, is it correct to say "we accept H1"? Answer is no, and your conclusion:

We have an evidence strong enough to believe that H0 is not true, but we may not have an evidence strong enough to believe that H1 is true. Therefore, "rejecting H0" does not automatically imply "accepting H1"

seems right to me.

Scientific theories are only built upon a certain set of propositions, until one of them is proven wrong. Along those lines the general idea of hypothesis testing is to rule out an immediate contradiction of a proposition by readily available facts, but it does not provide a proof of it.