# Is an F-test for equality of variance appropriate for a very large dataset?

I have a dataset with about 500,000 subjects and I am trying to establish whether the variance is equal. I first performed an F-test but then I realised the data is slightly skewed with kurtosis. So then I went with the Brown-Forsythe variation of the Levene test of variance because it utilises the median and thus is less influenced by non-normality in the data. Then I realised that, due to the central limit theorem, if the sample is sufficiently large, then one can treat the data as normally distributed.

So now I am torn. Do I perform the F-test or the Levene's test? Or is there a better test to carry out on data this size?

• Is it at all possible that the variances are exactly equal? With so much data, are both variants of the test significant? Why are you testing this in the first place? – gung - Reinstate Monica Jan 21 '15 at 18:32
• It isn't clear what purpose is served, or if it matters (but then I can't tell for sure from the information given). How much did the variances differ? It may help to read these excellent CV threads: Is normality testing 'essentially useless'?, & A principled method for choosing between t test or non-parametric e.g. Wilcoxon in small samples. – gung - Reinstate Monica Jan 21 '15 at 18:42
• Running a t-test may well have been fine. Did she run the Welch version? How much did the variances differ? – gung - Reinstate Monica Jan 21 '15 at 18:54
• I suspect the thread on testing large datasets at stats.stackexchange.com/questions/2516/… replies to the question that really ought to have been asked here. – whuber Jan 21 '15 at 19:36
• 1. "I am trying to establish whether the variance is equal" -- you can't establish equality. You can be pretty much certain the population variances are unequal. With a large enough n you'll reject equality, whether it actually matters or not. _ 2. "due to the central limit theorem, if the sample is sufficiently large, then one can treat the data as normally distributed" -- not so. If $n$ is very large, it may be that one can treat say the sample mean as normally distributed, but not the original data. If I have $10^9$ points from an exponential distribution, the distribution is still skew – Glen_b Jan 21 '15 at 23:05

A-priori, I find it highly implausible that the variances would be exactly equal (meaning that the null hypothesis should not be rejected, even with very high $n$ as you have here). As a result, I do not see what purpose is served by testing for heteroscedasticity*.
Assuming one prefers to use a $t$-test (to, say, the Mann-Whitney $U$-test), when heteroscedasticity exists, using the Welch correction may be appropriate. However, given that the ratio of variances is $1.072037$, and $N = 500,\!000$, I can't see that it matters much either way. I don't think the validity of the non-Welch-corrected $t$-test is threatened by heteroscedasticity so small, and I suspect the Welch $t$ would be virtually identical to the standard $t$ with that $N$.