11
$\begingroup$

When we do testing we end up with two outcomes.

1) We reject null hypothesis

2) We fail to reject null hypothesis.

We do not talk about accepting alternative hypotheses. If we do not talk about accepting alternative hypothesis, why do we need to have alternative hypothesis at all?

Here is update: Could somebody give me two examples:

1) rejecting null hypothesis is equal to accepting alternative hypothesis

2) rejecting null hypothesis is not equal to accepting alternative hypothesis

$\endgroup$
  • 1
    $\begingroup$ Because you are trying to draw some conclusions. If it is not the null hypothesis, then maybe it is the alternative hypothesis (even though you are not completely sure the alternative hypothesis is valid, if you reject the null hypothesis). When you reject the null hypothesis, you say that you have some "evidence" to conclude that the alternative hypothesis may be true. $\endgroup$ – nbro Jan 13 at 17:47
  • $\begingroup$ @nbro, thank you, I added question to my original post. Could you have a look? $\endgroup$ – user1700890 Jan 13 at 17:53
  • 1
    $\begingroup$ I am not super familiar with hypothesis testing, in general. It is better you wait for a more competent person to answer your questions. $\endgroup$ – nbro Jan 13 at 18:01
  • $\begingroup$ If your alternative hypothesis is a complement of null hypothesis, there's no point in using at all. Nobody uses alternative hypothesis in practice for this reasons outside textbooks. $\endgroup$ – Aksakal Jan 13 at 18:30
  • $\begingroup$ "We do not talk about accepting alternative hypotheses" -- not true for all possible "we". Some people do talk about accepting the alternative hypothesis, and many others think it, even if they respect the taboo against saying it. It is somewhat pedantic to avoid talking about accepting the alternative hypothesis when there is no reasonable doubt that it is true. But, since statistics is so prone to misuse, in this case the pedantry is probably a good thing in so far as it inculcates caution in the interpretation of results. $\endgroup$ – John Coleman Jan 14 at 15:36
7
$\begingroup$

I will focus on "If we do not talk about accepting alternative hypothesis, why do we need to have alternative hypothesis at all?"

Because it helps us to choose a meaningful test statistic and design our study to have high power---a high chance of rejecting the null when the alternative is true. Without an alternative, we have no concept of power.

Imagine we only have a null hypothesis and no alternative. Then there's no guidance on how to choose a test statistic that will have high power. All we can say is, "Reject the null whenever you observe a test statistic whose value is unlikely under the null." We can pick something arbitrary: we could draw Uniform(0,1) random numbers and reject the null when they are below 0.05. This happens under the null "rarely," no more than 5% of the time---yet it's also just as rare when the null is false. So this is technically a statistical test, but it's meaningless as evidence for or against anything.

Instead, usually we have some scientifically-plausible alternative hypothesis ("There is a positive difference in outcomes between the treatment and control groups in my experiment"). We'd like to defend it against potential critics who would bring up the null hypothesis as devil's advocates ("I'm not convinced yet---maybe your treatment actually hurts, or has no effect at all, and any apparent difference in the data is due only to sampling variation").

With these 2 hypotheses in mind, now we can setup up a powerful test, by choosing a test statistic whose typical values under the alternative are unlikely under the null. (A positive 2-sample t-statistic far from 0 would be unsurprising if the alternative is true, but surprising if the null is true.) Then we figure out the test statistic's sampling distribution under the null, so we can calculate p-values---and interpret them. When we observe a test statistic that's unlikely under the null, especially if the study design, sample size, etc. were chosen to have high power, this provides some evidence for the alternative.

So, why don't we talk about "accepting" the alternative hypothesis? Because even a high-powered study doesn't provide completely rigorous proof that the null is wrong. It's still a kind of evidence, but weaker than some other kinds of evidence.

$\endgroup$
7
$\begingroup$

There was, historically, disagreement about whether an alternative hypothesis was necessary. Let me explain this point of disagreement by considering the opinions of Fisher and Neyman, within the context of frequentist statistics, and a Bayesian answer.

  • Fisher - We do not need an alternative hypothesis; we can simply test a null hypothesis using a goodness-of-fit test. The outcome is a $p$-value, providing a measure of evidence for the null hypothesis.

  • Neyman - We must perform a hypothesis test between a null and an alternative. The test is such that it would result in type-1 errors at a fixed, pre-specified rate, $\alpha$. The outcome is a decision - to reject or not reject the null hypothesis at the level $\alpha$.

    We need an alternative from a decision theoretic perspective - we are making a choice between two courses of action - and because we should report the power of the test $$ 1 - p\left(\textrm{Accept $H_0$} \, \middle|\, H_1\right) $$ We should seek the most powerful tests possible to have the best chance of rejecting $H_0$ when the alternative is true.

    To satisfy both these points, the alternative hypothesis cannot be the vague 'not $H_0$' one.

  • Bayesian - We must consider at least two models and update their relative plausibility with data. With only a single model, we simple have $$ p(H_0) = 1 $$ no matter what data we collect. To make calculations in this framework, the alternative hypothesis (or model as it would be known in this context) cannot be the ill-defined 'not $H_0$' one. I call it ill-defined since we cannot write the model $p(\text{data}|\text{not }H_0)$.

$\endgroup$
  • 1
    $\begingroup$ Your last point is excellent, and often neglected in publications which base their whole argumentation on a single, unmotivated NHST. $\endgroup$ – Konrad Rudolph Jan 14 at 15:37
  • $\begingroup$ Why is 'not $H_0$' ill-defined? $\endgroup$ – Michael Apr 3 at 19:05
  • $\begingroup$ What is it? Can you calculate $p(data| not H0)$? $\endgroup$ – innisfree Apr 11 at 12:03
  • $\begingroup$ @innisfree under frequentist conception not, but probably under Bayesian. $\endgroup$ – Michael Jun 13 at 9:09
  • $\begingroup$ Try and do that without introducing at least 2 models... $\endgroup$ – innisfree Jun 14 at 2:30
4
$\begingroup$

Im am not 100% sure if this is a formal requirement but typically the null hypothesis and alternative hypothesis are: 1) complementary and 2) exhaustive. That is: 1) they cannot be both true at the same time ; 2) if one is not true the other must be true.

Consider simple test of heights between girls and boys. A typical null hypothesis in this case is that $height_{boys} = height_{girls}$. An alternative hypothesis would be $height_{boys} \ne height_{girls}$. So if null is not true - alternative must be true.

$\endgroup$
  • 1
    $\begingroup$ I completely agree with your statements, but one should note that both $H_0$ and $H_a$ are commonly infinitely large sets of null hypotheses. It also seems that many are convinced that $H_0$ and $H_a$ need not to be exhaustive, e.g. see this or this discussion. $\endgroup$ – bi_scholar Jan 13 at 20:39
  • 2
    $\begingroup$ @bi_scholar thank you for discussion threads. I am no expert in this but based on simple reasoning I do believe they have to be exhaustive. Consider this weird test: someone finds 5 rocks arranged in order on a road. His $H_0$: wind did this. His $H_1$: it was aliens. Now if he tests the chance that wind did this and finds a probability of 0.0001 - he rejects the wind hypothesis. But it doesn't give him the right to claim it was aliens. All he can claim is that the chance of it being wind is small. But ANY other explanation remains open. $\endgroup$ – Karolis Koncevičius Jan 13 at 23:16
  • 1
    $\begingroup$ I agree. My reasoning was that hypothesis testing is about accepting or rejecting $H_0$ while rejecting or accepting $H_a$. If $H_0$ and $H_a$ are not exhaustive, there is no point in defining any $H_a$ at all, since even when we reject $H_0$ we can not accept $H_a$, as there exist other hypotheses outside of $H_0$ and $H_a$ which might also be true. I unfortunately didn't manage to get my point across in the first thread. $\endgroup$ – bi_scholar Jan 13 at 23:25
  • 1
    $\begingroup$ @innisfree one could test two point hypotheses in some kind of likelihood framework - sure. But that procedure would't bet called "null hypothesis testing" and it's imprecise. It would select the closest one as being true even in cases when none of them are true. Furthermore regarding power - one can pick an alternative hypothesis or effect size when calculating power of the test but (in my view) should forget it once the testing is taking place. Unless there is some prior information that tells him about the possible effects present in the data. Like maybe white/black pixels in a noisy photo. $\endgroup$ – Karolis Koncevičius Jan 14 at 15:28
  • 1
    $\begingroup$ @innisfree I am curious how such a test would look like, could you formulate a small example? I'm convinced that we can not accept $\theta = 1$ by rejecting $H_0$ unless $\theta \in \{0, 1\}$ which corresponds to $H_0$ and $H_1$ being exhaustive. $\endgroup$ – bi_scholar Jan 14 at 17:06
2
$\begingroup$

Why do we need to have alternative hypothesis at all?

In a classical hypothesis test, the only mathematical role played by the alternative hypothesis is that it affects the ordering of the evidence through the chosen test statistic. The alternative hypothesis is used to determine the appropriate test statistic for the test, which is equivalent to setting an ordinal ranking of all possible data outcomes from those most conducive to the null hypothesis (against the stated alternative) to those least conducive to the null hypotheses (against the stated alternative). Once you have formed this ordinal ranking of the possible data outcomes, the alternative hypothesis plays no further mathematical role in the test.


Formal explanation: In any classical hypothesis test with $n$ observable data values $\mathbf{x} = (x_1,...,x_n)$ you have some test statistic $T: \mathbb{R}^n \rightarrow \mathbb{R}$ that maps every possible outcome of the data onto an ordinal scale that measures whether it is more conducive to the null or alternative hypothesis. (Without loss of generality we will assume that lower values are more conducive to the null hypothesis and higher values are more conducive to the alternative hypothesis. We sometimes say that higher values of the test statistic are "more extreme" insofar as they constitute more extreme evidence for the alternative hypothesis.) The p-value of the test is then given by:

$$p(\mathbf{x}) \equiv p_T(\mathbf{x}) \equiv \mathbb{P}( T(\mathbf{X}) \geqslant T(\mathbf{x}) | H_0).$$

This p-value function fully determines the evidence in the test for any data vector. When combined with a chosen significance level, it determines the outcome of the test for any data vector. (We have described this for a fixed number of data points $n$ but this can easily be extended to allow for arbitrary $n$.) It is important to note that the p-value is affected by the test statistic only through the ordinal scale it induces, so if you apply a monotonically increasing transformation to the test statistics, this makes no difference to the hypothesis test (i.e., it is the same test). This mathematical property merely reflects the fact that the sole purpose of the test statistic is to induce an ordinal scale on the space of all possible data vectors, to show which are more conducive to the null/alternative.

The alternative hypothesis affects this measurement only through the function $T$, which is chosen based on the stated null and alternative hypotheses within the overall model. Hence, we can regard the test statistic function as being a function $T \equiv g (\mathcal{M}, H_0, H_A)$ of the overall model $\mathcal{M}$ and the two hypotheses. For example, for a likelihood-ratio-test the test statistic is formed by taking a ratio (or logarithm of a ratio) of supremums of the likelihood function over parameter ranges relating to the null and alternative hypotheses.


What does this mean if we compare tests with different alternatives? Suppose you have a fixed model $\mathcal{M}$ and you want to do two different hypothesis tests comparing the same null hypothesis $H_0$ against two different alternatives $H_A$ and $H_A'$. In this case you will have two different test statistic functions:

$$T = g (\mathcal{M}, H_0, H_A) \quad \quad \quad \quad \quad T' = g (\mathcal{M}, H_0, H_A'),$$

leading to the corresponding p-value functions:

$$p(\mathbf{x}) = \mathbb{P}( T(\mathbf{X}) \geqslant T(\mathbf{x}) | H_0) \quad \quad \quad \quad \quad p'(\mathbf{x}) = \mathbb{P}( T'(\mathbf{X}) \geqslant T'(\mathbf{x}) | H_0).$$

It is important to note that if $T$ and $T'$ are monotonic increasing transformations of one another then the p-value functions $p$ and $p'$ are identical, so both tests are the same test. If the functions $T$ and $T'$ are not monotonic increasing transformations of one another then we have two genuinely different hypothesis tests.

$\endgroup$
  • 2
    $\begingroup$ I would agree with this, saying that the test is designed to reject the null hypothesis when faced with extreme results, and the role of the alternative hypothesis is to point at which results would be seen as extreme if the null hypothesis were true $\endgroup$ – Henry Jan 14 at 8:45
1
$\begingroup$

The reason I wouldn't think of accepting the alternative hypothesis is because that's not what we are testing. Null hypothesis significance testing (NHST) calculates the probability of observing data as extreme as observed (or more) given that the null hypothesis is true, or in other words NHST calculates a probability value that is conditioned on the fact that the null hypothesis is true, $P(data|H_0)$. So it is the probability of the data assuming that the null hypothesis is true. It never uses or gives the probability of a hypothesis (neither null nor alternative). Therefore when you observe a small p-value, all you know is that the data you observed appears to be unlikely under $H_0$, so you are collecting evidence against the null and in favour for whatever your alternative explanation is.

Before you run the experiment, you can decide on a cut-off level ($\alpha$) that deems you result significant, meaning if your p-value falls below that level, you conclude that the evidence against the null is so overwhelmingly high that the data must have originated from some other data generating process and you reject the null hypothesis based on that evidence. If the p-value is above that level you fail to reject the null hypothesis since your evidence is not substantial enough to believe that your sample came form a different data generating process.

The reason why you formulate an alternative hypothesis is because you likely had an experiment in mind before you started sampling. Formulating an alternative hypothesis can also decide on whether you use a one-tailed or two-tailed test and hence giving you more statistical power (in the one-tailed scenario). But technically in order to run the test you don't need to formulate an alternative hypothesis, you just need data.

$\endgroup$
  • $\begingroup$ NHST does not calculate $P(data|H_0)$; it calculates $P(\textrm{data as extreme as that observed}|H_0)$. The distinction is important. $\endgroup$ – innisfree Jan 14 at 9:32
  • $\begingroup$ @innisfree I agree and that's exactly how I defined data in that same sentence. $\endgroup$ – Stefan Jan 14 at 11:53
  • $\begingroup$ ? I can’t see anywhere where data is defined (that way or any other way) $\endgroup$ – innisfree Jan 14 at 12:12
  • $\begingroup$ And even if it were, why do that? Why redefine data that way? I’d advise to clarify the parts of the text around p(data.. $\endgroup$ – innisfree Jan 14 at 12:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.