Two things always bugged me about hypothesis testing:
Please, anyone can shed some light?
In frequentist hypothesis testing it is meaningless to talk about "the chance that the population mean is a given number" because the population mean is a fixed but unknown value. In particular, frequentist testing does not assume that the population mean is a random variable and hence it is meaningless to talk about $P(\mu=0)$.
The alternative hypothesis matters in the selection of the critical region which is the set of realizations of the test statistic that would imply a rejection of the null in favor of the alternative. For example, if you specify the alternative as $\mu >0$ then you would use a one-tailed test instead of a two-tailed test.
You may reject the null hypothesis but you never accept it, you only fail to reject it. That is, you may conclude that the evidence (observations) is not sufficiently strong to reject the null hypothesis, but you do not embrace the null hypothesis and accept it.
For example, in a clinical trial to test whether a certain medicine is efficacious, the null hypothesis is that the medicine is not effective. If the evidence is strong that the medicine is effective, you reject the null. If the evidence is weak, you say that there is not sufficient evidence to reject the null hypothesis. You do not declare thst the medicine is ineffective (accept the null), just that there is not enough evidence to say that it is effective (do not reject the null). In the case of a point null such as $\mu = 0$, you can say with some confidence that $\mu \neq 0$ if the evidence points that way, but in the presence of weak evidence, a savvy statistician would say that there is not sufficient evidence to conclude that $\mu \neq 0$ rather than proclaim to all the world that $\mu = 0$ as proven by the test just concluded. After all, the actual value of $\mu$ might be ever so slightly different from $\mu\ldots$
When Fisher first devised what is now called hypothesis testing, he didn't have an alternative hypothesis in mind. He simply wanted to create a statistic that measured the degree of agreement between the estimate and a proposed value. He found the probability of getting a value for an estimator further away from the proposed value than the estimate from the data. The p-value is just a one-to-one transformation of the test statistic. No alternative hypothesis here.
It was Neyman and Pearson that created the null and alternative hypothesis formulation and embedded it within decision theory---which of these statements should I accept? (I'm using "accept" a bit loosely here.) They wanted to find a procedure that was correct as often as possible (thus linking the concept to the frequentist notion of repeated sampling). They chose to minimize the chance of failing to reject a false null (minimize the Type II error or maximize power) for a given chance of rejecting a true null (for a given probability of a Type I error). This framework required the statement of a null hypothesis to determine the chance of rejecting a true null (which is the p-value, same as Fisher calculated) and the statement of the alternative hypothesis to find the procedure that is most powerful in detecting the alternative when it is true. Typically, we can't find a test that is the most powerful against all possible alternatives for a given null; restated, the alternative matters in the choice of the test.
So you do use the alternative when you do hypothesis testing: it is baked in to the test that you choose to use in the first place.
While it is common to always write the null hypothesis using only an equals sign ($\mu=\mu_0$) in truth the null hypothesis contains all the values not included in the alternative hypothesis, so in fact if we have $H_a: \mu > \mu_0$ then the null that we are testing is really $H_0: \mu \le \mu_0$. Even the 2-tailed test null hypothesis is really that the true value of the mean is in a small interval around the claimed null value, that interval is determined by the level of rounding in the measurement and recording of the data and the precision of the computer.
