2
$\begingroup$

In hypothesis testing, the common approach is to first set a null hypothesis and a hypothesis we want to test. Then apply some statistical techniques and see whether the observation is likely to happen under the assumption If null hypothesis is true. If the likelihood is low we will then reject the null hypothesis and claim our assumption is true.

But why we are not doing this the other way round? That is setting only the assumption we want to test and look at the observation. By applying some statistical technique I believe we can get how likely the observation happens under the assumption/hypothesis. We can then just use this probability to accept/reject hypothesis.

Why we are not using the second approach? It seems indirect to me in the first approach.

$\endgroup$
3
$\begingroup$

No matter how many white swans I see, I can never confirm the hypothesis that all swans are white. However, the single observation of a non-white swan can falsify my hypothesis.

This is the hallmark of scientific theories. They can never be confirmed, only falsified. So how does this apply to statistics?

We can consider our null hypothesis as a scientific hypothesis. When I do an experiment, compute a test statistic, and reject the null hypothesis, what I am saying is something to the effect of

I have made an assumption of how the world truly is. I have also obtained data that, under my assumption of how the world truly is, would be incredibly unlikely. I am now presented with either 1 of 2 options: either accept that I have observed an incredible improbably event, or conclude that my assumption of how the world truly is may be false.

We often opt for the second option. Since we have chosen to believe that our assumption of how the world truly is is false, we reject that hypothesis.

I'm sure many philosophers of statistics and science are rolling in their graves, but I rather like the thought of statistics being like science (only in so far as hypotheses are falsifiable and not verifiable).

$\endgroup$
1
$\begingroup$

The second approach is also used. For example, you want to test, if your sample is from gamma distribution family. This would be your null hypothesis $H_0,$ and if you don't set alternative hypothesis $H_1$ (such as "sample is from beta distribution family"), it would be automatically set as "sample is not from gamma distribution family". The problem is if you choose such a general alternative, the power of your test will decrease (probability of type II error will increase).

$\endgroup$
1
$\begingroup$

Yes, you are asking a lucid question. How does it make sense to assess the data while assuming the null hypothesis is true? Shouldn't we rather use the data to evaluate the hypotheses? Yes, the latter makes much more intuitive sense.

The former idea --- what we often do --- is null hypothesis significance testing (NHST). You can read up on the history of NHST, and in what ways it is controversial.

As to the question, I'm not qualified to answer, so you can consider this some idle conjecture. But I think some of the reasons why we use NHST is that some of the tests are well known, have been in use for a relatively long time, and are relatively easy to conduct. I also think that if you understand what the results mean, that they are useful. In my opinion, a lot of the criticism of NHST comes down to the fact that people often misunderstand the results of the hypothesis tests.

You might look up Bayesian analysis, which could be thought of as a way to assess hypotheses based on the observed data. This makes intuitive sense, but is less straightforward to conduct than, say, something like a t-test.

$\endgroup$
0
$\begingroup$

You can do it the other way around using "almost sure hypothesis testing". An almost sure hypothesis test rejects the null whenever it is false and accepts the null whenever it is true with probability one in any sufficiently large sample, which is tantamount to setting the significance level to n^{-p} p>1 where n is the sample size.It is fixing the type I error that causes the asymmetry. There is also a paper by Dembo and Peres that discusses treating null and alternative symmetrically.

$\endgroup$
0
$\begingroup$

One reason is that alternative hypothesis are not well-formed in that they don't describe a specific state of the world; rather they express the negation of the specific state described in the null hypothesis. For example, in performing an a correlation test between two variables, one typically sets the null hypothesis to be that the population correlation $\rho$ is $0$, a specific value, while the alternative is that $\rho$ is not $0$, a huge range of values. The problem is that we can't compute the probability of observing the data we found under the alternative hypothesis because the alternative hypothesis doesn't describe a specific state of the world.

If you proposed a specific alternative hypothesis (e.g., that $\rho$ was equal to $.25$), you would run into the same problem whereby no amount of evidence could ever confirm that hypothesis; the hypothesis can only be rejected. In this way, it's equivalent to just setting the null hypothesis to be that the correlation is $.25$.

One important method worth considering is confidence intervals. A confidence interval is the region in the parameter space for which, if we had set the null hypothesis value to a value outside the region, we would reject the null hypothesis. A confidence interval allows one to reject many null hypotheses, not just the primary one of interest. For example, if we find a confidence interval for $\rho$ of $.1$ to $.2$, we can reject any null hypothesis that posits a value of $\rho$ less than $.1$ or greater than $.2$. This isn't the same as proving a specific alternative hypothesis, but it does dramatically expand the null hypotheses that can be assessed with one data set, granting more scientific knowledge from a single analysis than simply rejecting one null hypothesis.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.