Why do we find evidence against null hypothesis in hypothesis testing?

Question

Why do you make the claim that you want to prove ( or have a hunch to be true) to be alternative hypothesis?

One might argue that it has to do with the way hypothesis testing is set up. That you find evidence against null hypothesis as based on the definition:
A null hypothesis is a statistical hypothesis that is tested for possible rejection under the assumption that it is true.

So the subsequent question is: Why do we find evidence against null hypothesis? Why not against alternative?

Answer 1

Because we are making an assumption that the null hypothesis is true, and are trying to prove that assumption by computing the probability a random sample follows that assumption. (And also the alternate hypothesis is the exact opposite of the null hypothesis.)

Answer 2

The answer by micro5 is right stating that probability computations are made assuming the null hypothesis. If we observe something that is very unlikely under the null hypothesis, this counts as evidence against the null hypothesis (note that in fact what should be very unlikely is what we observed or something that is even further away from what is expected under null hypothesis; in many situations whatever we observe is very unlikely to observe precisely, so this in itself would not be evidence against the null).

As the probability computations that decide about the p-value, or whether the test rejects or does not reject, are computed assuming the null hypothesis, these cannot provide evidence against the alternative. The alternative only comes into play in the sense that the test is defined is such a way that if we reject the null, the data actually point in the direction of the alternative. If the alternative is indeed true, the probability is high (or at least higher than under the null) to reject the null.

Tests are not symmetric in null hypothesis and alternative; they are in the first place constructed to have a low type I error probability (of rejection) if the null is indeed true.

In fact, not only the null hypothesis can never be "proved", neither can the alternative. We're talking about probability models, and as G. Box wrote, "all models are wrong but some are useful". All tests have model assumptions. These go into the definition of both null and alternative, and these can never be guaranteed to hold.

Note also that almost always alternatives are composite, meaning that you may for example test $H_0:\ \mu=0$, where $\mu$ is a normal mean, against $H_1:\ \mu>0$. So even if we reject the $H_0$ and believe in the alternative (which we shouldn't, see above), the test doesn't tell us what $\mu$ is; under the alternative it can be anything larger than 0.