# Statistical test to compare means when I have two observations per group?

I am trying to interpret the results of my experiment but am not sure what statistical method I should choose. For my experiment, I added 3 types of different fertilizers to soil, with a control, and measured their growth. I only had two replicates per treatment, so my data looks like this:

Treatment   Replicate  Length
Control       1          16
Control       2          18
Fert1         1          20
Fert1         2          17
Fert2         1          25
Fert2         2          27
Fert3         1          23
Fert3         2          21


I think people usually do a minimum of 3 replicates to calculate a "meaningful" standard deviation, but the logistics made it just not possible to do so. In this case, if I want to compare the means of each treatment, and determine whether a specific treatment significantly enhanced or repressed growth, what would be the best method to use? Can I use ANOVA with two observations per group? Would it be better to use the t-test between Control and Fert1 (or Fert2 or Fert3) separately to evaluate effect of each fertilizer to conrol? Would any of these statistical tests (or other tests I'm not aware of) be heavily biased toward having only two observations per group?

• The problem is not bias but power. With too small samples there is hardly any chance to find a significant difference even if a true difference exists. Furthermore there is no way to meaningfully check model assumptions. Nov 6 '21 at 0:51

You can run an ANOVA and then do a post-hoc comparison. With only two observations per group there needs to be decent size differences between the groups. You can also run Dunnet's Test to compare all three against the control. See code below.

There is no minimum rule sample size such as 3. All depends on the effect size you're trying to detect.

library(tidyverse)

a1 = tibble(treatment=c("Control","Control","Fert1","Fert1","Fert2","Fert2","Fert3","Fert3"),
length=c(16,18,20,17,25,26,23,21))

a1 %>% group_by(treatment) %>% summarize(mean(length),sd(length))

fit=aov(length~treatment,data=a1)
TukeyHSD(fit)

library(DescTools)
DunnettTest(x=a1$$length, g=as.factor(a1$$treatment))

• Thank you for your suggestion! I ran your code, but when I printed fit out, it gives a 2*2 table, with Sum of Squares and Deg. of Freedom as rows and treatment and Residuals as columns. This looked very different from other tables I googled, which has variables like treatment as rows and Df, Sum sq, Mean Sq, F value, Pr as columns. Did I make a mistake while I was running this?
– Jen
Nov 8 '21 at 19:29
• Try summary(fit)
– Glen
Nov 8 '21 at 22:56

First, the sample variance of a sample of size 2 may not be the best estimate of the population variance, but it does exist, so your model with 2 replications per level of a one-factor experiment should work according to the usual formulas. in R:

y = c(16, 18);  var(y)
[1] 2


x = c(16,18, 20,17, 25,27, 23,21)
g = as.factor(c(4,4, 1,1, 2,2, 4,4))


Notice that for procedures in R that use lm for ANOVA require that the group vector be declared as.factor.

Standard one-factor ANOVA in R gives results as follows:

anova(lm(x~g))

Analysis of Variance Table

Response: x

Df Sum Sq Mean Sq F value  Pr(>F)
g          2 71.375  35.688  5.0264 0.06359 .
Residuals  5 35.500   7.100
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1


R also implements an ANOVA that does not require equal variances among population levels. The denominator degrees of freedom are reduced according as sample variances among the levels differ (in this instance decreased from 5 to 2.68).

oneway.test(x ~ g)

One-way analysis of means
(not assuming equal variances)

data:  x and g
F = 9.1961, num df = 2.0000, denom df = 2.6839, p-value = 0.06294


The Kruskal-Wallis nonparametric test, which is somewhat analogous to a one-factor ANOVA, also works for your data, provided there are not (many) ties in the x vector.

kruskal.test(x~g)

Kruskal-Wallis rank sum test

data:  x by g
Kruskal-Wallis chi-squared = 4.125, df = 2,
p-value = 0.1271

• Thank you for these suggestions! Just to clarify, when you defined g, did you mean to set it as c(4,4, 1,1, 2,2, 3,3) to reflect different treatments (instead of having two pairs of 4,4)?
– Jen
Nov 8 '21 at 19:23
• Also, I looked up the Kruskal-Wallis test and the major difference between that and the one-way ANOVA test seemed to be that the former assumes data is not normally distributed, while the latter assumes a normal distribution. Since I only have two data per treatment (not enough data to check for normal distribution), how should I decide which distribution (normal or not normal) is a more accurate description of my data?
– Jen
Nov 8 '21 at 19:25
• Unless you have reason to believe data are not normal, I would be comfortable assuming normality. The differences between the two replicates in each group are modest and about the same, which is consistent with normal and even with equal variances. Also, the K-W test has low power with such small samples. So I'd use one of the ANOVAs. // If this is all the data you're going to have for now, you won't have significance at the 5% level, but significance at the 10% level, which might encourage a more extensive look. // (As for K-W: It does not assume normal data, but works OK with normal data.) Nov 8 '21 at 21:16