Is it correct to claim "equivalence" when there is no significant difference between two methods? Say we are interested in  assessing the effectiveness of two different methods on improving some measure $x$.
We randomly assign a group of 40 individuals to method 1 and a group of 40 individuals to method 2. We take a measurement pre intervention and post intervention. Following the intervention, we conduct a 2 samples t-test and get a p-value of 0.078. Is it correct to claim that both approaches are equally effective? I have seen this in a paper and don't agree that this is the correct interpretation (at least not when based on the p-value alone).
My perspective
I know NHST effectively gives us the following: $p(\text{data} \mid H_0)$. With that in mind, I do not think that claiming equivalence solely on the basis of an insiginificant p-value is strictly correct.
Why? Well, first of all, the hypothesis test answers the question "are the differences observed large enough to be considered surprising, assuming the null is true". It does not answer the question "are the differences exactly 0". In other words, we can observe differences >0 OR <0 and still obtain an insignificant p-value. Clearly this does not mean the difference in the two methods is =0 OR that the methods are equivalent. Now let's just say that in our example, that group 1 experiences a larger change in measure $x$  than group 2 (and we see this reflected by effect size or in the raw change scores), and yet we observe an insignificant p-value of 0.078. I can understand why some make the argument that the two methods can more or less be considered "equivalent" (for lack of a better term), because the true difference may well be 0 and observed difference is likely due to sampling variation, measurement error, or some combination of both. BUT, it is also possible that this is a type II error: If we had a larger sample size, would this difference be significant? Yes, at some-point it would.
So, for me, claims of equivalence should be based on either (a) inferiority/equivalence hypothesis testing or (b) as a minimum, interpretation of effect sizes/ magnitude of change scores, and NOT just on the basis of p-values. Yes, if the effect is large enough it will still be significant with smaller samples, but my point pertains more to using the p-value alone to make this claim.
I just find the claim of equivalence quite bold and strong when it is based solely on the magnitude of a p-value. I do, however, accept that this claim has more weight if the researcher has purposefully conducted an equivalence hypothesis test. What does everyone else think? Am I way off? Is this a fair comment or not?
 A: As "equivalence testing" has a well-established meaning in statistics, you are correct that failure to reject a null hypothesis should not lead to a claim that the treatments are "equivalent" or "equally effective." Such a claim is properly based on a pre-chosen maximum difference that can be taken to be "equivalent" in practice, based on knowledge of the subject matter. As Walker and Nowacki ("Understanding Equivalence and Noninferiority Testing," J. General Internal Medicine 26: 192–196, 2011) emphasized:

The determination of the equivalence margin, $\delta$, is the most critical step in equivalence/noninferiority testing... It must be stressed that the value of the equivalence margin should be determined before the data is recorded. This is essential to maintain the type I error at the desired level.

You can't simply use the results of a standard t-test to establish equivalence post-hoc. They go on to start the next section of their review:

NO DIFFERENCE DOES NOT IMPLY EQUIVALENCE
Using a traditional comparative test to establish equivalence/noninferiority leads frequently to incorrect conclusions. The reason is two-fold. First, the burden of the proof is on the wrong hypothesis, i.e., that of a difference. In this setting, a significant result establishes a difference, whereas a nonsignificant result implies only that equivalency (or equality) cannot be ruled out. Consequently, the risk of incorrectly concluding equivalence can be very high. The other reason is that the margin of equivalence is not considered, and thus the concept of equivalence is not well defined.

The literature is clearly on your side here.
