Hypothesis test for comparing means of many groups, with unequal variances?

Question

I am looking for a hypothesis test to compare the means of many groups similar to ANOVA or Kruskal–Wallis. The problem is my data does not meet the necessary conditions for these tests having significantly different variances (I tested this with Bartlett's Test) meaning I cannot use ANOVA and the data is not continuous so I cannot use Kruskal–Wallis.

Other Characteristics of the Data:

• 6 Groups
• Discrete Data
• 100 measurements per group
• all groups are roughly normal
• each has a different mean and variance

It is also worthwhile to note I will want to do some post hoc tests to find if the greatest result is significant compared to the rest, and therefore if there any tests that I can perform to directly compare the group with the highest mean to the rest that my data would work well with, that would also be helpful. Ultimately I want to find if a given group, or multiple groups, is significantly greater than the rest.

Answer 1

In R the test implemented in oneway.test does not assume equal variances. In somewhat the same way that the Welch 2-sample t test accommodates 2 groups with possibly different variances, this ANOVA procedure handles $k \ge 2$ groups.

Here is a demonstration with three groups. Groups have 10 replications each and are sampled from populations $\mathsf{Norm}(\mu=100,\sigma=5), \mathsf{Norm}(\mu=110,\sigma=10),$ and $\mathsf{Norm}(\mu=130,\sigma=20),$ respectively.

set.seed(2020) # retain this line to get the same data I used
x1 = rnorm(10, 100, 5);  x2 = rnorm(10, 110, 10);  x3 =rnorm(10, 130, 20)
x = c(x1,x2,x3);  g = rep(1:3, each=10)
oneway.test(x ~ g)

        One-way analysis of means (not assuming equal variances)

data:  x and g
F = 36.369, num df = 2.000, denom df = 15.183, p-value = 1.621e-06

Notice the reduced denominator DF. A standard ANOVA would have DF(Resid) = 27.

If the null hypothesis is rejected (as in this example), you can use Welch t tests for ad hoc comparisons---along with the Bonferroni (or other) method of avoiding false discovery for multiple tests.

In my example, there is no significant difference between groups 1 and 2.

t.test(x1,x2)

        Welch Two Sample t-test

data:  x1 and x2
t = -1.7237, df = 11.782, p-value = 0.1109
alternative hypothesis: 
  true difference in means is not equal to 0
95 percent confidence interval:
 -21.560090   2.535926
sample estimates:
mean of x mean of y 
 99.48019 108.99227 

But there is a highly significant difference between groups 2 and 3.

t.test(x2,x3)$p.val
[1] 9.651986e-05

Boxplots give an overview of the data.

boxplot(x ~ g, col="skyblue2", pch=20)