3
$\begingroup$

I stumbled on the quirk given below.

I am working on two implementations, one to test for equality of means using the t-test and another testing for equality of variances using Brown-Forsythe's F-test.

Then I notice that the results are exactly the same!

I have only two groups. I get the same result if I assume equality of variance in the t-test and if I use the mean, not the median, in the F-test (so it really is the Levene test).

Since I use only two groups, the F-test's $\nu_1 = 1$. Now remember when $t \sim t( \nu_2 )$ then $t^2 \sim F( 1, \nu_2 )$.

So I assume the critical $t$ for the means test, squared, is the same as the critical $F$ of the variance test and looking at the formulae it indeed appears to be so.

I also realise that it makes sense that testing for equality of means' deviation, when squared, is the same as the sum of squared deviations from the common mean that is used to test for equality of variances if there are only two groups.

The quirkiness is that the critical value of an equality of means test is the same as the critical value of an equality of variances test.

Based on Dave's excellent comments, I'd like to ask the following.

Suppose I have two data sets that I want to test for equality. I am really interested in testing the means, but it would be interesting to test the variances as well.

My question is if the methodology I propose below is correct.

Since the means test can be done under the assumptions of equal or unequal variances, I test the variances first using Levene (means not medians). The test shows me the variances are not significantly different, so I proceed to test the means under the assumption of equal variances in the t-test.

Does it make sense to do it this way? Especially since I notice that both tests yield the exact same result due to reasons described above.

$\endgroup$
  • $\begingroup$ The subject matter is great--but on this site you have to ask a definite question! $\endgroup$ – whuber Feb 7 at 19:24
  • $\begingroup$ Thanks I appreciate the feedback. I also really benefited from the comment I got from Dave before and have used that and your remark to turn this into a question. I hope it is now complying. $\endgroup$ – skaak Feb 8 at 6:25
3
$\begingroup$

What you’ve discovered is that equal-variance two-sample t-testing is equivalent to the ANOVA F-test of two groups. This is a known fact, but certainly take a few minutes to feel good about discovering it on your own.

What your post title makes it sound like you’re doing is testing for a difference in the means of two groups and testing for difference in the variances of two groups and doing those the same way. That is incorrect. You can use the F-test to compare the variances of two groups, but then you’re not comparing the means. That’s okay. Sometimes variances are interesting to compare.

But comparing the variances of two groups is different from comparing the “within” and “between” variances of those two groups, which is inference about their difference in mean.

Remember, analysis of variance uses a test of variance but is a tool for inspecting for differences in mean.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks Dave it helps a lot. I've also rephrased it a bit to turn it into a question and would appreciate you to respond to that please. $\endgroup$ – skaak Feb 8 at 6:22
  • $\begingroup$ @skaak No, what you propose doesn’t make sense. You’re running the same test twice in different ways, only examining for differences in the group means, and you don’t test the equality of the variances of the groups. I’ll expand on this later. Until then, see what happens if you apply this method to two groups with equal means but different variances. Your variance test should reject, but it won’t (consistently), because you’re not testing the variances you think you’re testing. $\endgroup$ – Dave Feb 8 at 12:01
  • $\begingroup$ Thanks, I think I get it ... and you are correct in that I was running the same test twice. $\endgroup$ – skaak Feb 8 at 18:12
0
$\begingroup$

The answer to this lies in Levene's test for $y_{ij}$ coming from $i=1..k$ groups $$ F = {{n-k}\over{k-1}} \cdot { { \Sigma n_i(z_{i\cdot}-z_{\cdot\cdot})^2 } \over { \Sigma\Sigma(z_{ij}-z_{i\cdot})^2 } } $$

The important thing here is that

$$ z_{ij} = | y_{ij} - \bar y_i | $$

so the answer is to standardise twice in Levene's formula above.

First you standardise the $y$ and then you standardise the $z$ and only then do you apply the test.

If you omit this little detail, and you have only two groups, then remarkably you end up repeating the t-test for reasons described in the question and pointed out in Dave's answer and comments.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.