Should we teach and use equivalence tests given that we have confidence intervals already?

Question

This question is inspired by the discussion here. The background is that often point null hypotheses like $\mu=0$ are tested, $\mu$ being some effect of interest in some suitably parametrised model, and people argue that in reality $\mu=0$ precisely will often not occur, but in terms of interpretation $|\mu|\le\epsilon$ for some sufficiently small $\epsilon$ is basically the same as saying that "there is no effect" (meaning then "any existing effect is so small as to not be of practical relevance). A well known issue with hypothesis tests is that in such a case, when testing $H_0:\ \mu=0$ and indeed $\mu=\frac{\epsilon}{2}$, say, a large enough data set will reject $H_0$, as the test has then enough information to see that $\mu\neq 0$ even though very small.

Equivalence tests test a null hypothesis of the kind $|\mu|\le\epsilon$ with pre-specified $\epsilon$, and it has been argued that because of the above mentioned defect of standard tests, people should use and teach equivalence tests instead (which of course require to specify $\epsilon$).

Another approach would be to just on top of the standard test compute a confidence interval and see whether something as small as $\epsilon$ is in it. I think people should generally not only be interested in rejecting (or not rejecting) a null hypothesis, but also in effect sizes, so computing a confidence interval and interpreting it seems to be generally good advice. Given that this is so, I wonder what equivalence testing actually adds. Are there any reasons to run an equivalence test given that we can well address the same issue using a confidence interval (the confidence interval has the additional advantage that $\epsilon$ doesn't need to be specified, and one can rather look at the CI and see whether something is in there that seems "too small to be meaningful" a posteriori)? And even if situations can be constructed in which the equivalence test adds something worthwhile, could these situations be so exotic and exceptional that it basically isn't worth the hassle to teach and advertise them big time?

PS: Addition prompted by @FrankHarrell's answer: Certain Bayesians think that neither of these should be taught or used, which within the Bayesian paradigm is fair enough. I disagree (though I'm not against Bayes in general), but the question is obviously set within a frequentist framework (not necessarily claiming that this should dominate; even if you favour Bayes in most situations you may use frequentist approaches from time to time), and is not intended to have a Bayesians vs. frequentist discussion here.

Answer 1

As a Bayesian, I am puzzled why the frequentist way of thinking is still the default. With Bayesian posterior inference you don't need special cases or different study designs to answer questions about superiority vs. questions about similarity/"equivalence". You just compute more posterior probabilities, and you capitalize on directionality. A single study can provide evidence for superiority, similarity, and harm, and can also address multiple response variables. Examples: P(effect > 0), P(effect > 5%), P(effect < -2%), P(effect on response 1 > 2% and effect on response 2 > 3%).

In general computing a probability that the effect is in an investigator-specified interval is more useful than finding an uncertainty interval attached to a given investigator-specified coverage probability.

Answer 2

Thinking about it, I could see two advantages:

• In pre-registered studies, one may then choose $\epsilon$ a priori, based on certain arguments, such that $|\mu|<\epsilon$ would may be deemed uninteresting but anything above would be perceived as most likely relevant/meaningful. This makes it harder to move the goalpost if the effect does not have the expected strength.
• The value of $\epsilon$ is also a basis for choosing a sample size that gives the test the proper amount of statistical power. So, specifying $\epsilon$ beforehand may help planning the study anyway. This may help counter the argument according to which the null will always be rejected provided the sample size is large enough (by making it just large enough to find interesting effects, but not much larger than that).

This makes null hypothesis testing a little closer to ideal Popperian falsification, by committing to stricter requirements for rejecting the null prior to testing, and by making the null hypothesis a more challenging contender to the alternative.

Now is your question that, even if one accepts to commit to a certain choice of $\epsilon$ prior to the study, then CI remains more informative and should be preferred? If this is about being more informative then, I agree with @Frank Harrell's Bayesian take, that the posterior distribution of the effect size would be more interesting anyway.