I am extremely confused about getting the between group variance component from an ANOVA. To my understanding, variance within groups ($V_w$) equals the mean of squares within groups ($MSW$) and the variance between groups ($V_b$) can be calculated from the mean of squares between groups ($MSB$) as following $MSB = MSW + k*V_b$, where $k$ is the number of observations per level of the grouping variable.

However, trying to follow this logic by hand or in R I am not able to get the variances right. Let's consider the following example with a subset of mtcars with $k=4$.

m4 = mtcars %>% dplyr::filter(cyl == 4) %>% head(n=4)
m6 = mtcars %>% dplyr::filter(cyl == 6) %>% head(n=4)
m8 = mtcars %>% dplyr::filter(cyl == 8) %>% head(n=4)
df = dplyr::bind_rows(m4, m6, m8)

In the dataframe above, the total variance of mpg (var(df$mpg)) is $21.95$.

If I use ANOVA now, the variances I calculate are:

aov(data = df, mpg ~ factor(cyl)) %>% summary()
#             Df Sum Sq Mean Sq F value  Pr(>F)   
# factor(cyl)  2 160.86   80.43   8.984 0.00717 **
# Residuals    9  80.57    8.95                   
# ---
# Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

$V_w = 8.95$, and $V_b = (80.43-8.95)/4 = 17.87$, consequently, the total variance estimate is $Vw + Vb = 26.82$. But this value is larger than the one above. Why the difference? am I getting something wrong?

Also a couple of questions in case anybody knows. If I had different number of observations per level (e.g. as in the original mtcars data set, what would be the "$k$" parameter multiplying $V_b$ in $MSB$ ? Would it be possible to estimate $V_b$ in a similar way for a non-parametric design using Kruskal-Wallis?


  • 1
    $\begingroup$ I find it striking that $26.82 / 21.95 = 11/9.$ $\endgroup$
    – whuber
    Nov 30, 2021 at 22:37
  • $\begingroup$ Thanks for the pointer I guess? but still confusing. If in the example above (Vw + Vb)*(9/11) = 21.95, which is the total variance. But if one repeats the example with k = 3 and changing the quotient of degrees of freedom to 6/8, it does not work. I'd appreciate a source where I could read how to compute $V_b$, if what you are suggesting is that the above expression for MSB is wrong. $\endgroup$ Nov 30, 2021 at 22:58

1 Answer 1


Here is an approach to a balanced one-factor ANOVA that may have mnemonic value. Consider the model $Y_{ij} = \mu_i + e_{ij},$ where $e_{ij} \stackrel{iid}{\sim}\mathsf{Norm}(0,\sigma).$

Suppose we have $h = 3$ groups and $k=10$ replications for each group then DF.Fact $= h-1 = 2$ and DF.Resi $=h(k-1)=3(9) = 27.$

One says that there are two estimators of the common variance $\sigma^2:$ One is the "pooled" variance $S_w^2$ of the three groups: $S_w^2 = (S_1^2 + S_2^2 + S_3^2)/3,$ in the balanced case. It is a good estimator of $\sigma^2.$ whether or not the three groups have the same population mean.

A second estimator $S_b^2$ of $\sigma^2$ is unbiased only if all three group means are equal to $\mu,$ otherwise it tends to be too large. Thus if all group population means are $\mu,$ we have $E(\bar X_1) = E(\bar X_2) = E(\bar X_3) = \mu.$ Then also $Var(\bar X_1) = Var(\bar X_2) = Var(\bar X_3) = \sigma^2/k.$ And the sample variance $S_{\bar X_i}^2$ of the three group means estimates $\sigma^2/k.$ Then $S_b^2 = kS_{\bar X_i}^2$ estimates $\sigma^2.$

The F-statistic is MS.Factor/MS.Resi $S_b^2/ S_w^2.$ Large values of the F-statistic lead to rejection of the null hypothesis that all group means are equal.

Consider fictitious data for this design and their resulting ANOVA table in R, as follows:

x1 = rnorm(10, 50, 7)
x2 = rnorm(10, 55, 7)
x3 = rnorm(10, 60, 7)
x = c(x1,x2,x3)
g = as.factor(rep(1:3, each=10))
Analysis of Variance Table

Response: x
          Df  Sum Sq Mean Sq F value   Pr(>F)   
g          2  520.01 260.004  6.1835 0.006153 **
Residuals 27 1135.29  42.048                    
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Now notice that we can get $S_w^2$ or MS.Resi = 42.048 of the ANOVA table as follows:

[1] 42.04788

Also, we can get $S_b^2$ or MS.Factor = 260.004 of the ANOVA table as follows:

[1] 260.0037

There are similar, but somewhat messier, formulas for $S_b^2$ and $S_w^2$ if the design is not balanced.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.