I have a question about a data analysis that I am running. I am analyzing the results of a survey in which (expectedly) there exists far fewer people in one group than in another. This survey is an Honours project about enhancement drug use, and I have ~40 users and ~590 non-users. Obviously there is a huge difference between the sizes of these groups! I am interested in differences in means on a particular study processes scale. I have the following questions:

  1. Is it a problem to use an ANOVA in this case (where the groups are so different in sample size)?

    I know that the Levene's test is sensitive to differences in sample size (such that it is far more likely to be significant); and this is the case in my data, p < .05. Regardless, Welch and Brown-Forsythe tests still reveal a statistically significant differences in mean scores on this scale.

  2. Given that the Welch and Brown-Forsythe tests still reveal significant differences between groups, would I be wrong in concluding that these do in fact exist (despite violating the initial Levene's assumption)?

  3. Is there are more robust way to investigate this research question, taking into account the disparity in group sizes?


It is ok if your groups have different sample sizes as long as each has a minimum number of cases in them (min 30 cases each).

If your Leven's test is significant then inequality of variances are assumed and you should use the P value and assess its significance under unequal variances.

You can calculate and consider the effect size when comparing the two means which takes into account the sample size effect... your other option is to try to down sample your big group to an equal size and then re assess the group differences.

  • 5
    $\begingroup$ (-1) The claim that one needs a minimum of 30 cases per group is false. More discussion at this answer and other ones for the same question: stats.stackexchange.com/a/129888/121522 $\endgroup$
    – mkt
    Jul 25 '17 at 12:00
  • $\begingroup$ The underlying assumption of many statistical methods, including regression and ANOVA is the normality assumption...I mentioned sample size 30, according to central limit theorem (CLT). instead of quickly down voting with not much grounds, If you read the post you've shared yourself carefully, you will see that the author mentions about the power of the test as well as type I error. Of course you can run Anova with a sample size smaller than 30, however you're contradicting the underlying assumptions of these tests and hence increasing the chance of type I error that is an unreliable results! $\endgroup$
    – RomRom
    Oct 27 '17 at 4:15
  • 4
    $\begingroup$ There is a difference between saying (1) sample size is an important consideration and (2) you need 30 cases per group. The first is certainly true, the second is nonsense. You seem to think I'm arguing against the central limit theorem, but I'm instead arguing against the existence of a specific threshold besides the trivial one described in the linked answer. And as for reading the post I shared - the number 30 appears nowhere in the question, answers or comments. $\endgroup$
    – mkt
    Oct 27 '17 at 8:06

It doesn't matter what is the underlying distribution of the Dependent variable if you are using Levene's test to check variance because Levene's Test uses Median instead of Mean so it works well on the Skewed Data. So if the p-values is significant you can assume that the variances are different between the classes and proceed with the ANOVA test. Also, it doesn't matter if the Class distribution is so different.

Levene's method is same as Standard Deviation except it has Median in place of Mean.

  • 1
    $\begingroup$ This doesn't seem to be an accurate description of the standard Levene's test. $\endgroup$ Aug 6 '18 at 22:53

Not the answer you're looking for? Browse other questions tagged or ask your own question.