1
$\begingroup$

I did a Standard Error Plot of my two treatments, and homogeneity of variance appeared to be justified (there was only a slight difference in variance).

However, then I ran Levene's during my independent t-test and got a Sig. value of 0.000.

My sample size was in the thousands for each treatment, so I was wondering how does sample size affect the outcome of Levene's? My assumption is that: as sample size increases, the accuracy of the sample in relation to the population increases and therefore only a slight difference in variance would be really magnified.

Some clarification on this would be awesome, thanks!

$\endgroup$
1
$\begingroup$

Your intuition is correct. As with all tests, a higher $n$ will lead to higher power all other things being equal. A larger actual difference in variances obviously also makes it more likely that the Levene test can reject the null hypothesis. But with large $n$, even effects (here difference in variance) that are practically too small to worry about become statistically significant.

You will also note that using the Levene test this way (and I don't really know another way to use it) brings a fundamental conflict of interest: You want to prove the null hypothesis! You do this Levene test to conclude equality of variances, not to reject it. But you shouldn't try to prove the null hypothesis. Trying to prove the null hypothesis rewards lazy data collection or in your case penalizes abundant data.

If your primary goal is to learn about those variances, your best shot are equivalence testing or confidence interval based approaches. With a high $n$, you will have rejected $H_0 \quad \sigma_1=\sigma_2$, but you can also get an upper- and lower bound of that difference from the confidence interval. The larger your $n$, the more precisely you will know how much difference there is in variances.

It is more likely that you did that test as a precursor to another test that requires equality of variances. Know that unequal variances are only really a problem when the sample sizes are very different as well. Also, for t-tests for example you could just use the Welch version and not worry about variances at all.

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ I see! Thank you very much. So simply, the S.E. plot showed there was little difference in variance because samples may have differed by a few units. However, as a result of the higher n, Levene's puts the 8000 measurements together and sees that first sample has a few units of difference from the other 8000, and basically concludes - if the experiement ran 8000 times, then if the samples shared the same population they would have been closer to one another. I apologize for simplistic terms, I am just trying to wrap my head around it. $\endgroup$ – Bartholomas Sep 19 '17 at 13:36
  • $\begingroup$ "So simply, the S.E. plot showed there was little difference in variance because samples may have differed by a few units." By a few units of variance you mean? Because the Levene test can reject equal variances also when the number of sample points in both groups is exactly equal. The rest of your comment looks like you understood. $\endgroup$ – David Ernst Sep 19 '17 at 13:41
0
$\begingroup$

What affects Levene's-Test Sig. value?

  1. differences in spread in the sample value

  2. sample size, at least when the null is not strictly true (as with any consistent test)

My sample size was in the thousands for each treatment,

This is why tiny differences are significant ... and it's also why hypothesis tests of assumptions of procedures make little sense. They answer the wrong question. The relevant question isn't "is this assumption true" (it almost certainly isn't, not exactly), it's "how much difference does it make?" and that's not what the hypothesis test tells you about. It takes a fairly substantial difference in variance to badly impact the analysis in a really appreciable way; with large samples you'll detect trivial differences.

Correspondingly failure to reject in a test of assumptions is also of little comfort if you lack power to detect differences that matter to the original analysis.

There's also the problem that choosing between procedures on the basis of the test of hypothesis impacts the properties of the procedures you're choosing among.

A diagnostic plot that lets you see how different the spreads are is a more useful indicator than a test (it shows you how different they are, which is somewhat closer to what you need to know)

An equivalence test -- if you could identify suitable bounds on what would be reasonable to call equivalent would make more sense, but it would still have an issue with the "choosing between procedures impacts their properties" part.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.