# Which statistical test is applicable if the variance within groups is large?

I have multiple groups, each group with 3 replicates so I performed a Kruskal-Wallis One Way Analysis of Variance on Ranks since normality was not achieved; however, post-hoc Tukey tests show no significant differences. The problem is that there is a lot of variance within each group so no significant difference is found, ideally I would have more replicates but this is not possible now so I was wondering if anyone could suggest an alternative statistical test. Thank you!

• It sounds like you have an underpowered test no matter what. Subtle differences are harder to detect than major differences, and large variability in the data will cause even fairly large differences to be subtle. Now what do you mean by replicates, samples? What’s making you call them replicates instead of the more usual terminology? I can think of situations where this is totally reasonable but want to hear straight from you. – Dave Mar 28 at 17:44
• Tukey is not an appropriate ad hoc procedure for K-W. Maybe use Mann-Whitney-Wilcoxon tests, with Bonferroni as protection against false discovery. // Tukey OK as ad hoc for one-way ANOVA. – BruceET Mar 28 at 18:10
• Yh I was worried that might be the case Dave and I sampled 3 times at 8 separate sites so I just used the term replicate as I was repeating the sampling procedure in the same area. I was dredging so the procedure was not always in the exact same location but more of a general site and the method is obviously affected by dredge efficiency which changes with each sample so maybe sample is a better term? – Keelan Priscott Mar 29 at 11:22
• Cheers BruceET! Will give it a go – Keelan Priscott Mar 29 at 11:23

Here are simulated data that may be somewhat similar to yours. There are six groups with three observations per group.

set.seed(2020)         # for reproducibility of data
x1 = rnorm(3, 100, 2)
x2 = rnorm(3, 100, 4)
x3 = rnorm(3, 110, 2)
x4 = rnorm(3, 110, 4)
x5 = rnorm(3, 115, 2)
x6 = rnorm(3, 115, 4)
x = c(x1, x2, x3, x4, x5, x6)
gp = as.factor(rep(1:6, each=3))


In spite of different variabilities among groups, a Kruskal-Wallis test finds a significant difference (P-value 0.016) among group locations.

kruskal.test(rank(x)~gp)

Kruskal-Wallis rank sum test

data:  rank(x) by gp
Kruskal-Wallis chi-squared = 14.006, df = 5, p-value = 0.01557


Although there are only three observations per group, there are 18 observations altogether, and a detectable pattern of increasing location values as we move from Group 1 through Group 6.

However, a two-sample Wilcoxon rank sum test finds no significant difference even between the (perhaps maximally different) locations of Groups 1 and 6.

wilcox.test(x1,x6)

Wilcoxon rank sum test

data:  x1 and x6
W = 0, p-value = 0.1
alternative hypothesis: true location shift is not equal to 0


Here is a plot of all 18 observations with vertical lines separating the six groups. Even though all Group 6 values exceed any Group 1 value (complete separation), there are only $${6 \choose 3} = 20$$ partitions of six objects into two groups, with two instances of complete separation among them, so the minimum attainable P-value is $$1/10 = 0.1.$$ [If we assume normality (inappropriate for your data) and do a Welch 2-sample t test, the P-value is 0.0055 with no protection against false discovery. Tukey multiple comparisons assume normality and homoscedasticity.]

x1; x6
 100.75394 100.60310  97.80395
 119.7855 113.5137 114.5070


The bottom line is that using nonparametric rank-based tests, you would need at least four observations per group in order to have a chance of seeing significant differences (at the 5% level) in location between groups---even without Bonferonni (or other) protection against false discovery in ad hoc analysis.

The fundamental problem with your data is not the differences among variances, mentioned in the title of your question; it is the fact that you have only three observations per group. I guess this is essentially what @Dave meant in his Comment: "It sounds like you have an underpowered test...."

Note: An alternative to the Kruskal-Wallis test might have been to do a ("Welchified") one-way ANOVA on the ranks of the 18 observations in data vector 'x'. Ranks of nonnormal data would not be normal, but they could not have troublesome outliers. (The P-value, 0.01 is not much different from that of the K-W test.)

oneway.test(rank(x)~gp)

One-way analysis of means (not assuming equal variances)

data:  rank(x) and gp
F = 10.922, num df = 5.0000, denom df = 5.3648, p-value = 0.008129