Four catalysts that may affect the concentration of one component in a three-component liquid mixture are being investigated. Consider a completely randomized experiment, where $n_1 = 5$, $n_2 = 4$, $n_3 = 3$, and $n_4 = 4$. Suppose that prior to running the experiment the investigator wanted to make the following comparisons:

(a) Average of Catalyst 1 and 2 vs Average of Catalyst 3 and 4

(b) Catalyst 1 vs Catalyst 2

(c) Catalyst 3 vs Catalyst 4.

Answer: In a balanced design I understand that in order to be a contrast the condition $\sum\limits_{i = 1}^a c_i = 0$ needs to be satisfied, where the $c_i$ are the coefficients of the $\mu_i$ in the contrasts. So for example, in a balanced design the contrast for Part (c) would be $\Gamma = \mu_3 - \mu_4$.

In an unbalanced design the condition changes to $\sum\limits_{i = 1}^a n_ic_i = 0$. But I'm confused. $\Gamma = \mu_3 - \mu_4$ no longer works. So do we have to make up coefficients so that the condition is satisfied? For example, for Part (c), knowing that $n_3 = 3$ and $n_4 = 4$, then $\Gamma = 4\mu_3 - 3\mu_4$ would be a contrast.

Is this correct?


1 Answer 1


There are not enough residuals here (16) for a useful check of normality. And you say nothing about whether group variances are equal. You say nothing about checking assumptions, so I'm wondering whether unequal sample sizes are the only challenge here. If normality and homoscedasticity assumptions are warranted, then of course you could do a standard ANOVA.

Here are simulated normal data with unequal standard deviations.

x1 = rnorm(5,20,3);  x2 = rnorm(4,25,2)
x3 = rnorm(3,45,3);  x4 = rnorm(4,40,4)
x = c(x1,x2,x3,x4)
g=as.factor(c(1,1,1,1,1, 2,2,2,2, 3,3,3,4, 4,4,4))

Stripcharts give an overview of the simulated data. (With sample sizes 5 and below it seems inappropriate to use boxplots.)

stripchart(x ~ g, pch=20, ylim=c(.5,4.5))

enter image description here

Assuming normal data, equal variances. A standard ANOVA finds strong evidence that population means are unequal.

anova(lm(x ~ g))

Analysis of Variance Table

Response: x
          Df  Sum Sq Mean Sq F value    Pr(>F)    
g          3 2134.79  711.60  63.235 1.267e-07 ***
Residuals 12  135.04   11.25                      
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Then ad hoc tests (b) and (c) can be done directly, using pooled t tests. For my simulated data, Groups 1 and 2 differ significantly, but groups 3 and 4 don't.

t.test(x1,x2, var.eq=T)$p.val
[1] 0.003257274
t.test(x3,x4, var.eq=T)$p.val
[1] 0.3804501

And you can combine groups to test (a), also with pooled t tests. The first two groups differ from the second two.

t.test(c(x1,x2), c(x3,x4), var.eq=T)$p.val
[1] 3.458586e-07

To avoid false discovery, you could use the Bonferroni method, testing at the 1.67% level, which is achieved for both significant results.

Assuming normal data, not necessarily equal variances. In R, the procedure oneway.test implements a version of ANOVA that adjusts DF(Resid) as necessary to compensate for different sample variances. (The approach is the same as for the Welch t test--extended to more than two groups.) Then ad hoc tests can be done using Welch two-sample t tests. Results are essentially the same as when group variances are (here wrongly) assumed equal.

oneway.test(x ~ g)

        One-way analysis of means (not assuming equal variances)

data:  x and g
F = 55.357, num df = 3.0000, denom df = 5.8291, p-value = 0.0001112

[1] 0.003708631
[1] 0.3492354

Not assuming normal data. Here we use the same pattern of analysis, but using nonparametric tests: Kruskal-Wallis for all four groups, and two-sample Wilcoxon tests for ad hoc comparisons. For my simulated data the findings are the same, but P-values are somewhat smaller, perhaps reflecting the smaller powers of the nonparametric tests.

kruskal.test(x ~ g)

        Kruskal-Wallis rank sum test

data:  x by g
Kruskal-Wallis chi-squared = 13.335, df = 3, p-value = 0.003966

[1] 0.01587302
[1] 0.4
wilcox.test(c(x1,x2), c(x3,x4))$p.val
[1] 0.0001748252

Comments. If you are sure about normality and equal population variances, then you may want to pursue the use of contrasts. (Also, if this happens to be a textbook exercise on contrasts.) If not, you should use alternative procedures to compare the four groups, and use ad hoc procedures that share assumptions with those procedures.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.