I know the means, sample sizes, and standard deviations of three samples drawn independently from three populations. I need to test whether they differ significantly from each other using an $F$ statistic. However, I've looked at a few Stats books and can't find a procedure to do this. I'm familiar with simple ANOVAs and with testing for the difference of two means, if that helps to understand what sorts of things I already understand.

  • $\begingroup$ Many elementary and intermediate-level applied statistics books give the formulas necessary to do the F-test in a one-way ANOVA. However, it is possible to do the F-test if you have the 'sufficient statistics', which consist of the three sample sizes, the three sample means, and the three sample variances. My Answer gives elementary versions of the formulas (for equal sample sizes) and shows how to do the F-test using only the sufficient statistics. $\endgroup$ – BruceET Dec 6 '18 at 9:16

You want to do a one-factor analysis of variance (ANOVA) with three levels of the factor. (A one-factor ANOVA is sometimes called a one-way ANOVA.)

I will talk about the easiest version in which the sample sizes (numbers of 'replications') are the same for the three groups: $n_1 = n_2 = n_3= n.$ Also, one assumes that that the data in the three groups are normally distributed, possibly with different means $\mu_1, \mu_2,$ and $\mu_3,$ but all having the same variance: $\sigma_1^2 = \sigma_2^2 = \sigma_3^2 = \sigma^2,$ which is unknown.

Step 1: Within groups variance estimate: The first estimate of $\sigma^2$ is the average of the three sample variances: $\hat\sigma_w^2 = (S_1^2 + S_2^2 + S_3^2)/3.$ This estimate is valid whether or not the null hypothesis $H_0: \mu_1 = \mu_2 = \mu_3$ is true. It is called the 'within-groups' variance (estimate).

Step 2: Among groups variance estimate: The second estimate of $\sigma^2$ takes $H_0$ as true. If all three group means are equal, then each $\bar X_i \sim \mathsf{N}(\mu, \sigma^2/n).$ Then you treat the three $\bar X_i$'s as 'data' and find their variance. The 'grand mean' is the average $\bar{\bar X}=(\bar X_1 + \bar X_2 + \bar X_3)/3.$ Then the variance of the three sample means is

$$V = \frac{1}{3-1} \sum_{i=1}^2 (\bar X_i - \bar{\bar X})^2.$$

This $V$ is an estimate of $\sigma^2/n,$ so that $\hat\sigma_a^2 = nV$ is another estimate of $\sigma^2.$ Thus the 'among groups' estimate of $\sigma^2$ is

$$ \hat\sigma_a^2 = nV = \frac{n}{2}\sum_{i=1}^2 (\bar X_i - \bar{\bar X})^2.$$

This estimate of $\sigma^2$ will be accurate as long as $H_0$ is true; otherwise it will tend to be too large.

Step 3: Variance ratio and test. Finally, if $H_0$ is true, the ratio $F = \frac{\hat\sigma_a^2}{\hat\sigma_w^2}$ has an F-distribution with numerator degrees of freedom $\nu_a = 2$ and denominator degrees of freedom $\nu_w = 3(n-1):$ sometimes written as $\mathsf{F}(2, 3(n-1)).$ Roughly speaking, if $H_0$ is true, then $F \approx 1$ and if $H_0$ is not true, then $F >> 1.$ Tables of the F-distribution or statistical software are used to decide for each $n$ what 'critical value' separates "about 1" from "significantly larger than 1."

Note: If the number of replications differs from one group to another, the corresponding formulas (given in many statistics textbooks) are slightly more complicated. But it is still enough to know the sample sizes, means, and variances to do the computations necessary to complete an ANOVA. (In order to check for normality and equal variances, you would need the actual data.)

Example: Suppose you have data with $n = 10$ replications in each group, as shown below. Then let's see how to find the variance ratio and perform the desired test to judge whether the three group populations means are equal.

Generate illustrative data using R. In simulating the samples, I have used different population means: $\mu_1 = 100,\,$ $\mu_2 = 103,\,$ $\mu_3 = 110.$ When we finish the ANOVA we will see whether $n = 10$ replications in each group provide enough information to detect these differences in population means and thus reject $H_0.$

set.seed(1218);  n = 10
x1 = round(rnorm(n, 100, 3))
x2 = round(rnorm(n, 105, 3))  
x3 = round(rnorm(n, 110, 3))
x1; x2; x3
[1] 100  99 100 102 104 100 108 102 101 101  # 1st group
[1] 106 103 103  96 102 105 100 109 104 108  # 2nd
[1] 106 114 111 102 110 109 112 112 108 111  # 3rd
boxplot(x1, x2, x3, col="skyblue2")

enter image description here

Use R as a calculator to find the three sample means $(\bar X_1 = 101.7,\,$ $\bar X_2 = 103.6,\,$ $\bar X_3 = 109.5)$ and sample variances $(S_1^2 = 6.90,\, S_2^2 = 14.49,\, S_3^2 = 12.05).$ You might want to follow along with your own calculator.

Notice that the sample sizes, sample means, and sample variances are sufficient to complete the ANOVA.

a1;  a2;  a3
[1] 101.7
[1] 103.6
[1] 109.5
v1 = var(x1);  v2 = var(x2);  v3 = var(x3)
v1; v2; v3
[1] 6.9
[1] 14.48889
[1] 12.05556

If you already have the three sample means and the three sample variances, you can start here. Use R to find $S_a^2 = 165.43, S_w^2 = 11.148,$ and finally the variance ratio $F = 14.84.$

var.among = n*var(c(a1, a2, a3)); var.among
[1] 165.4333
var.within = mean(c(v1, v2, v3));  var.within
[1] 11.14815
f = var.among/var.within;  f
[1] 14.83953

Using R, other software, or printed tables of the F-distribution, you can find the critical value $c= 3.35$ that cuts 5% from the upper tail of $\mathsf{F}(2, 27).$ Because $F = 14.84 > c = 3.35$ we can say that the null hypothesis $H_0$ is rejected, and that there are significant differences among the three population means.

c = qf(.95, 2, 27);  c
[1] 3.354131

Using R, it is possible to find the P-value (very nearly 0) of this F-test. [P-values are 'creatures' of the computer age. Typically, there is not enough information in printed tables of the F-distribution to find exact P-values.]

1 - pf(14.84, 2, 27)
[1] 4.48857e-05

enter image description here

Note: Traditionally, some of the computations leading to the F-test in an ANOVA are displayed in an ANOVA table. The ANOVA table from Minitab for our data is shown below. The estimates $\hat \sigma_a^2$ and $\hat \sigma_w^2$ are labeled MS(Factor) and MS(Error), respectively, in this table.

Analysis of Variance

Source  DF      SS      MS  F-Value  P-Value
Factor   2   330.9  165.43    14.84    0.000      # That is, P-value < 0.0005
Error   27   301.0   11.15
Total   29   631.9
  • 1
    $\begingroup$ Thanks so much! Is there perhaps a typo where you wrote $\hat{\sigma}_b$? $\endgroup$ – Addem Dec 6 '18 at 12:46
  • $\begingroup$ Good catch. Fixing typo. Many books refer to variance "between" means; some (including me, except in typos) prefer the more grammatically correct "among". So you might see both notations if you look in various books. $\endgroup$ – BruceET Dec 6 '18 at 18:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.