How is statistical equivalence testing different from post hoc tests done after ANOVA?

  • 1
    $\begingroup$ What do you mean by "equivalence testing"? $\endgroup$ – chl May 26 '14 at 8:45
  • $\begingroup$ @chl "equivalence testing" is a thing. See my answer. $\endgroup$ – Alexis May 26 '14 at 16:38
  • 2
    $\begingroup$ @Alexis I'm aware of TOST or statistical tests of non-inferiority in randomized clinical trials. I thought it would be clearer if the OP could clarify in plain English what he/she wants to do. $\endgroup$ – chl May 26 '14 at 19:48

The post hoc tests following an ANOVA are pairwise t tests for mean difference. They have null hypotheses that the population means are equal (alternatively that the difference in population means is zero). Also, there are some post hoc details like pooled variance and multiple comparisons adjustments. We can set those aside, as they are largely irrelevant to your question.

By contrast, tests for equivalence (which could be, but are not necessarily post hoc tests following an ANOVA), have null hypotheses that the population means are different by at least some researcher-defined amount. You might get something out of reading the tost tag description.

One can combine the inferences from difference and equivalence tests, as I point out in my question here.

  • $\begingroup$ So you suggest equivalence testing to infer which pairs did not contribute to the global difference revealed by an ANOVA test? Interesting thought! In the end it's a completely different approach to the same interpretation "pairs A and B contribute to the revealed global difference, but pairs C and D are to similar". But I'm not sure if it is correct. $\endgroup$ – Horst Grünbusch Jun 4 '14 at 9:20
  • $\begingroup$ That's a fair point. Mostly, I was trying to spell out the differences between post hoc tests for difference, and equivalence tests. :) $\endgroup$ – Alexis Jun 4 '14 at 16:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.