4
$\begingroup$

I have a dataset with about 500,000 subjects and I am trying to establish whether the variance is equal. I first performed an F-test but then I realised the data is slightly skewed with kurtosis. So then I went with the Brown-Forsythe variation of the Levene test of variance because it utilises the median and thus is less influenced by non-normality in the data. Then I realised that, due to the central limit theorem, if the sample is sufficiently large, then one can treat the data as normally distributed.

So now I am torn. Do I perform the F-test or the Levene's test? Or is there a better test to carry out on data this size?

| cite | improve this question | |
$\endgroup$
  • 1
    $\begingroup$ Is it at all possible that the variances are exactly equal? With so much data, are both variants of the test significant? Why are you testing this in the first place? $\endgroup$ – gung - Reinstate Monica♦ Jan 21 '15 at 18:32
  • 1
    $\begingroup$ It isn't clear what purpose is served, or if it matters (but then I can't tell for sure from the information given). How much did the variances differ? It may help to read these excellent CV threads: Is normality testing 'essentially useless'?, & A principled method for choosing between t test or non-parametric e.g. Wilcoxon in small samples. $\endgroup$ – gung - Reinstate Monica♦ Jan 21 '15 at 18:42
  • 1
    $\begingroup$ Running a t-test may well have been fine. Did she run the Welch version? How much did the variances differ? $\endgroup$ – gung - Reinstate Monica♦ Jan 21 '15 at 18:54
  • 1
    $\begingroup$ I suspect the thread on testing large datasets at stats.stackexchange.com/questions/2516/… replies to the question that really ought to have been asked here. $\endgroup$ – whuber♦ Jan 21 '15 at 19:36
  • 1
    $\begingroup$ 1. "I am trying to establish whether the variance is equal" -- you can't establish equality. You can be pretty much certain the population variances are unequal. With a large enough n you'll reject equality, whether it actually matters or not. _ 2. "due to the central limit theorem, if the sample is sufficiently large, then one can treat the data as normally distributed" -- not so. If $n$ is very large, it may be that one can treat say the sample mean as normally distributed, but not the original data. If I have $10^9$ points from an exponential distribution, the distribution is still skew $\endgroup$ – Glen_b Jan 21 '15 at 23:05
5
$\begingroup$

A-priori, I find it highly implausible that the variances would be exactly equal (meaning that the null hypothesis should not be rejected, even with very high $n$ as you have here). As a result, I do not see what purpose is served by testing for heteroscedasticity*.

Assuming one prefers to use a $t$-test (to, say, the Mann-Whitney $U$-test), when heteroscedasticity exists, using the Welch correction may be appropriate. However, given that the ratio of variances is $1.072037$, and $N = 500,\!000$, I can't see that it matters much either way. I don't think the validity of the non-Welch-corrected $t$-test is threatened by heteroscedasticity so small, and I suspect the Welch $t$ would be virtually identical to the standard $t$ with that $N$.

* To understand this topic more thoroughly, it may help to read these two excellent CV threads:
  1. Is normality testing 'essentially useless'?
  2. A principled method for choosing between t test or non-parametric e.g. Wilcoxon in small samples.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.