1
$\begingroup$

Consider $X\sim N(1,1)$ and $Y\sim exp(1)$. The distributions are totally different, but they do have equal variance, so if I were to sample from these and test for unequal variance, I would not want to reject a null hypothesis of variance equality.

I am finding that the usual suspects for variance testing reject too often. In my Monte Carlo simulations, the F-test (var.test in R), Brown-Forsythe (car::leveneTest(…, center="median")), Levene (car::leveneTest(…, center="mean")), Ansari-Bradley (ansari.test), and permutation testing (my own custom function) all rejected considerably more often that my chosen $\alpha$. These tests are overpowered for this situation. I suspect (particularly for Ansari-Bradley) that this is due to the tests examining some kind of surrogate for variance (some kind of "scale" parameter) rather than variance itself, and they are finding that $X$ and $Y$ do differ on that surrogate.

What would be a test that has appropriate power for my $X$ and $Y?$

Improve this question
$\endgroup$
9
  • $\begingroup$ The choice of test ought to depend on the amount of data and on any additional assumptions you can make about the distributions. Could you provide some specifics of your intended application? $\endgroup$
    – whuber ♦
    Commented Dec 1, 2020 at 15:12
  • $\begingroup$ For amount of data, I won’t be working with 3-4 observations. I would expect (usually) at least dozens of not hundreds of points. For intended application, I have bizarre empirical distributions that don’t seem to follow the named distributions. Mostly, I’m content to use something like Ansari-Bradley (a collaborator adores it), but I’d sure like to see something that really examines variance and not the “scale” that Ansari-Bradley examines, since that results in a false rejection of variance equality for $N(1,1)$ and $exp(1)$. $\endgroup$
    – Dave
    Commented Dec 1, 2020 at 15:29
  • $\begingroup$ For comparing variances you typically need much more data than for comparing means. In this light, even a few hundred values is not a large dataset. The A-B test is a rank sum test, which means it may be appropriate for your broad nonparametric assumption: but it also implies it may lack power. The power can be regained to the extent you can narrow your characterization of the distributions your are comparing. I don't follow your distinction between a scale parameter and variance, because the two are interchangeable in any scale family (which this is). $\endgroup$
    – whuber ♦
    Commented Dec 1, 2020 at 15:51
  • $\begingroup$ The normal $X$ and exponential $Y$ are in the same scale family? $\endgroup$
    – Dave
    Commented Dec 1, 2020 at 16:01
  • $\begingroup$ Yes, because you have implicitly formulated this as a nonparametric problem. If that's not what you mean--if instead you literally mean you know you have sampled from this Normal and this Exponential distribution--then there's nothing left to say. In fact, you don't even need a sample to determine whether these distributions have equal variances! $\endgroup$
    – whuber ♦
    Commented Dec 1, 2020 at 16:06

1 Answer 1

Reset to default
0
$\begingroup$

In this case the means of the two distributions are equal. So instead of using a variance test, you can test the hypothesis $E[X^2]=E[Y^2]$.

Improve this answer
$\endgroup$
5
  • $\begingroup$ This is an interesting idea, but it turns out to give too many type I errors, even when I subtract each variable’s mean from itself to work with zero-mean empirical distributions. $\endgroup$
    – Dave
    Commented Dec 1, 2020 at 13:00
  • $\begingroup$ The OP hasn't suggested they would assume the means are equal. Regardless, how would you test equality of the expected squares? Remember, one explicit assumption is that "the distributions are totally different," so unless there is a huge amount of data, applying a standard t-test would be questionable. $\endgroup$
    – whuber ♦
    Commented Dec 1, 2020 at 15:11
  • $\begingroup$ Wilcoxon's test is designed for this. $\endgroup$
    – chrishmorris
    Commented Dec 2, 2020 at 9:41
  • $\begingroup$ Could you please explain how Wilcoxon's test handles this? Just square the values and throw them in wilcox.test in R? $\endgroup$
    – Dave
    Commented Dec 23, 2020 at 20:59
  • $\begingroup$ yes, pass in the two sets of (squared) values, with paired=False $\endgroup$
    – chrishmorris
    Commented Jan 1, 2021 at 13:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.