0
$\begingroup$

in university modules it is almost ritualistically taught that variances must be equal in different groups when performing, for example, a t-test or an ANOVA. I understand that the empirical p-value is calculated based on the assumption that the variance in all groups is equal. Thus, I can somehow comprehend that the violation of this assumption might have some unwanted effect.

However, I have no idea why this should be the case and how exactly the p-value is influenced. Neither did I find any literature about this issue. Can anyone help me out here? I would be very glad.

$\endgroup$
1
  • $\begingroup$ There is a huge literature about this. You can find a little bit of it by Googling. $\endgroup$
    – whuber
    Feb 14, 2020 at 17:49

1 Answer 1

1
$\begingroup$

In a t-test, for example, if your variances are unequal, this can affect the Type I error rate (i.e., rejection of a true null hypothesis). The way in which the significance level is affected can depend on group size. If group sizes are roughly equal, t-test and ANOVA will be robust to violation of this assumption. If large variances are associated with smaller groups, significance will be underestimated, which may mean that the null is falsely rejected. If large variances are associated with larger groups, significance would be overestimated. Also, see https://www.statisticssolutions.com/the-assumption-of-homogeneity-of-variance/

$\endgroup$
1
  • 1
    $\begingroup$ Are you quite sure about the association of variance size and group size? I think it is in the other direction (with small group size + large variance leading to overestimation of significance - because the pooled variance is then driven mainly by the larger group with smaller variance), at least that's what my simulations indicate. $\endgroup$
    – TJ27
    Oct 8, 2021 at 4:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.