5
$\begingroup$

I'm trying to compare two samples from multivariate normal distributions to see if their distributions are equivalent (within a factor of epsilon).

The standard version of this test is the energy test, but it is not useful for my purposes because it uses $P = Q$ as the null hypothesis, whereas I need $P \neq Q$ to be the null.

Some previous work has been done on this, mostly in this book, but nothing for my situation.

How can I extend the interval inclusion method described in this book to use the energy statistic? Should the test statistic used to get the confidence interval be the original test statistic (i.e. based on the null hypothesis $P = Q$), or do I have to derive my own based on the new null hypothesis? I think derive my own is definitely beyond my capabilities at this point.

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ If you look, say here it suggests looking at confidence intervals and avoiding p-values altogether. That sounds like sensible advice to me. $\endgroup$ – Glen_b Apr 6 '13 at 5:17
  • $\begingroup$ The problem with non-point nulls is you can't compute the distribution of the test statistic. Equivalence tests done using hypothesis testing only work if you have one-sided tests and then they're two normal tests, not the inequality type you mention. You can do the confidence interval equivalence thing, but again, it's not showing they're actually equal -- none of these things can do that. I presume you were the one asking about this stuff here? $\endgroup$ – Glen_b Apr 6 '13 at 5:26
  • 1
    $\begingroup$ So you want to show that $d[(\mu_1,\mu_2),(\mu'_1,\mu'_2)]<\epsilon$ ? (where $(\mu_1,\mu_2)$,$(\mu'_1,\mu'_2)$ are the two bivariate means). What is your choice of the distance $d$ ? Anyway this is straightforward with a noninformative Bayesian approach. $\endgroup$ – Stéphane Laurent Apr 6 '13 at 7:04
  • $\begingroup$ @Glen_b Yes, that was me. But I got the idea for confidence interval testing from here. The thing I'm confused about is where the confidence interval comes from. Can I use a regular test statistic, or do I have to come up with my own, since the null has changed? $\endgroup$ – user1871183 Apr 6 '13 at 17:07
  • $\begingroup$ @user1871183 Do you understand my previous comment ? This is an analogous statement of "equivalence testing" for bivariate data. $\endgroup$ – Stéphane Laurent Jun 5 '13 at 17:18
2
$\begingroup$

If you actually have the confidence interval, your first option is right. It's the TOST. Please remember to take the $1-2\alpha$-confidence interval to get an $\alpha$-level test. If this confidence interval is a subset of your prespecified equivalence region, you may conclude that the distributions are equal up to your prespecified error. Otherwise, you cannot conclude anything.

Improve this answer
$\endgroup$
-1
$\begingroup$

This sounds like bioequivalence, where the null hypothesis is "not equal". In that case, it is typically tested by two t-tests, one for the ratio being above 1.25, and one for the ratio being below 0.8.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.