# Good variance test when the distributions have different shapes but I really want to test the variance

Consider $$X\sim N(1,1)$$ and $$Y\sim exp(1)$$. The distributions are totally different, but they do have equal variance, so if I were to sample from these and test for unequal variance, I would not want to reject a null hypothesis of variance equality.

I am finding that the usual suspects for variance testing reject too often. In my Monte Carlo simulations, the F-test (var.test in R), Brown-Forsythe (car::leveneTest(…, center="median")), Levene (car::leveneTest(…, center="mean")), Ansari-Bradley (ansari.test), and permutation testing (my own custom function) all rejected considerably more often that my chosen $$\alpha$$. These tests are overpowered for this situation. I suspect (particularly for Ansari-Bradley) that this is due to the tests examining some kind of surrogate for variance (some kind of "scale" parameter) rather than variance itself, and they are finding that $$X$$ and $$Y$$ do differ on that surrogate.

What would be a test that has appropriate power for my $$X$$ and $$Y?$$

• The choice of test ought to depend on the amount of data and on any additional assumptions you can make about the distributions. Could you provide some specifics of your intended application?
– whuber
Commented Dec 1, 2020 at 15:12
• For amount of data, I won’t be working with 3-4 observations. I would expect (usually) at least dozens of not hundreds of points. For intended application, I have bizarre empirical distributions that don’t seem to follow the named distributions. Mostly, I’m content to use something like Ansari-Bradley (a collaborator adores it), but I’d sure like to see something that really examines variance and not the “scale” that Ansari-Bradley examines, since that results in a false rejection of variance equality for $N(1,1)$ and $exp(1)$.
– Dave
Commented Dec 1, 2020 at 15:29
• For comparing variances you typically need much more data than for comparing means. In this light, even a few hundred values is not a large dataset. The A-B test is a rank sum test, which means it may be appropriate for your broad nonparametric assumption: but it also implies it may lack power. The power can be regained to the extent you can narrow your characterization of the distributions your are comparing. I don't follow your distinction between a scale parameter and variance, because the two are interchangeable in any scale family (which this is).
– whuber
Commented Dec 1, 2020 at 15:51
• The normal $X$ and exponential $Y$ are in the same scale family?
– Dave
Commented Dec 1, 2020 at 16:01
• Yes, because you have implicitly formulated this as a nonparametric problem. If that's not what you mean--if instead you literally mean you know you have sampled from this Normal and this Exponential distribution--then there's nothing left to say. In fact, you don't even need a sample to determine whether these distributions have equal variances!
– whuber
Commented Dec 1, 2020 at 16:06

In this case the means of the two distributions are equal. So instead of using a variance test, you can test the hypothesis $$E[X^2]=E[Y^2]$$.

• This is an interesting idea, but it turns out to give too many type I errors, even when I subtract each variable’s mean from itself to work with zero-mean empirical distributions.
– Dave
Commented Dec 1, 2020 at 13:00
• The OP hasn't suggested they would assume the means are equal. Regardless, how would you test equality of the expected squares? Remember, one explicit assumption is that "the distributions are totally different," so unless there is a huge amount of data, applying a standard t-test would be questionable.
– whuber
Commented Dec 1, 2020 at 15:11
• Wilcoxon's test is designed for this. Commented Dec 2, 2020 at 9:41
• Could you please explain how Wilcoxon's test handles this? Just square the values and throw them in wilcox.test in R?
– Dave
Commented Dec 23, 2020 at 20:59
• yes, pass in the two sets of (squared) values, with paired=False Commented Jan 1, 2021 at 13:25