2
$\begingroup$

I think this is simple, and I am wrong-thinking it....

t-tests are used to test the null hypothesis that the means are equal;

$H_0 : \beta_1 = \beta_2$, $H_1 : \beta_1 \neq \beta_2$

But I want to test the other way around;

$H_0 : \beta_1 \neq \beta_2$, $H_1 : \beta_1 = \beta_2$

Do I just swap the hypotheses and 1-p the test result? or is there something subtle that I am missing?

$\endgroup$
1
  • 2
    $\begingroup$ No you can't just swap the hypotheses, unfortunately. And, generally, to get a workable test, the formulation gets modified a bit to be of the form $H_0: |\beta_1 - \beta_2| \geq \Delta$ vs. $H_1 : |\beta_1 - \beta_2| < \Delta$, for some prespecified $\Delta$. This arises often in bioequivalence testing. $\endgroup$
    – cardinal
    Sep 10, 2011 at 23:03

2 Answers 2

3
$\begingroup$

What you are talking about is called equivalence testing. The basic idea is to decide on a range that you consider equivalent (you need infinite data from each group to prove that they are exactly equal), then you find a confidence interval on the difference and if the CI falls completely within your equivalence range then you can call the 2 equivalent (the range of equivalence comes from the science not the statistics).

However note that this may not be exactly what you want. A population with mean 100 and standard deviation 1 and another population that only has values of 0 and 200 with 50% of each will have equal means, but I would not want to use one as a replacement of the other.

If you want to see if a cheaper test can be used in place of a gold standard (often what people want when they talk about wanting equivalence) then look at Bland-Altman techniques.

$\endgroup$
1
$\begingroup$

No, you can't just swap them around. The test is only designed to test for a difference. And the p-value itself has little to no meaning except for whether it exceeds the cut off for deciding whether you believe there is a statistically significant effect.

Furthermore, it's very unlikely that if your two conditions are subjected to some experimental treatment that there's not some sort of effect that perhaps you cannot measure. In this very likely case, attempting to prove B1 = B2 is pointless since it's not true (and some argue is never true).

That said, a common approach to your dilemma is to calculate a confidence interval for your effect. Then you make a reasoned argument about whether any observed differences are meaningful (as opposed to statistically significant). For example, let's say the score of group A is 9 and the score of group B is 11. You calculate a confidence interval for the observed effect of 2 and it ranges from -1 to 5. Now you set about and argue that if there is any effect it's probably less than 3 (not that there's no effect). Then you have to decide, if it were 3, or less, would it be meaningful anyway? And that's a discussion that you have to write for your particular case.

You can quantify the relatively likelihood of the Null and observed models by calculating likelihood ratios as well, or better yet, AIC scores. These will help you in any argument you have about whether the effect is so small it's trivial.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.