What is the inferential role of a point alternative hypothesis used for planning?

Question

Say I plan an experiment to test the null hypothesis of $H_0:\mu_1-\mu_2 = 0$ and, with a power analysis for a Student's $t$-test for independent samples, I find that samples of $n=10$ (per group) will give me power of 99.5% with the alternative hypothesis of $H_a: \mu_1-\mu_2=3$ with $\alpha=0.005$.

(Assume the statistical model is a good enough match to reality for the purposes of the analysis.)

Now I run the planned experiment and reject the null hypothesis. What is my inference regarding $H_a$?

It seems to me that any time I reject $H_0$ I implicitly accept the negation of $H_0$, but should I accept the planned point value of $H_a$?

A test where I accept the planning alternative whenever I reject the null would allow me to accept an alternative when the evidence do not actually favour it.

(Yes, there are several other questions an answers that are similar, but I have not found any that is explicit on this point, and I think clarity here will be useful.)

Answer 1

It seems you understand it just fine: evidence against null cannot be immediately translated into evidence for any particular $H_a$ (unless $H_a$ is just the negation of $H_0$). For an obvious example, a significant p-value from a two-tailed t-test tells you nothing about the effect direction.

If you need inference beyond null hypothesis testing, a small step forward is looking into confidence intervals, but the final answer is Bayesian analysis: then your posterior support for $H_a$ will be determined from prior support for $H_a$ and new evidence for it. In practice, this is often replaced with a qualitative discussion of probable effect sizes, power, and evidence ("we had good power to detect any effects larger than $c$..."). IMHO, this requires just as much effort and provides less answers than the Bayesian framework.