How to obtain the variance from an ANOVA table

Question

I was under the expression that the mean square error (MS) of a 1-way anova table is the estimated variance, and that the between and within group variance add to the total variance. However, using an example I get an result which puts me into doubts. Here the example:

I have a dataset in matlab.

data = [1.5377    0.6923    1.6501    3.7950    5.6715
    2.8339    1.5664    6.0349    3.8759    3.7925
   -1.2588    2.3426    3.7254    5.4897    5.7172
    1.8622    5.5784    2.9369    5.4090    6.6302
    1.3188    4.7694    3.7147    5.4172    5.4889];

If I use the anova1 function I obtain the anova table

From this I expected that the variance of the columns is 13.4309 and the variance of the random error is 2.2204. I also expected to obtain that the two MS values add to the total variance. However, if I use the variance function, I get the result

var(data(:)) 
>> 4.0888

This result differens significantly from the MS sum.

I know that

1. the sum-of-squares add up to the total sum-of-square (first column in picture) $$SS_{total} = SS_{columns} + SS_{error}$$
2. that the total variance is given by $$var_{total} = SS_{total}/df_{total}$$

However, I often read the following: "we are divide the total variance into a bewteen and a within group variance". I expected that this is actually true. However, from the above example and the two pieces of knowledge I conclude that it's actually the total sum-of-squares, which is seperated and not the total variance. Is this true or is there a way to seperate the total variance into two groups?

Answer 1

In the example we have $c = 5$ different groups (=columns) and each group consists of $n=5$ different data points. The $MS_{column}$ is given by $MS_{column} = n \cdot \sigma^2_{between}$. Thus, we have to divide $MS_{column}$ by the sample size per group to obtain the "between group variance".

Here the R code:

data.val = c(1.5377, 0.6923, 1.6501, 3.7950, 5.6715, 
             2.8339, 1.5664, 6.0349, 3.8759, 3.7925,
             -1.2588, 2.3426, 3.7254, 5.4897, 5.7172,
             1.8622, 5.5784, 2.9369, 5.4090, 6.6302,
             1.3188, 4.7694, 3.7147, 5.4172, 5.4889)
data.grp = c(rep(1:5, 5))

df = data.frame(val = data.val,
                grp = factor(data.grp) )

library(ggplot2)
gg = ggplot(df, aes(x = grp, y = val, fill=grp)) + 
    geom_boxplot(alpha=0.5) +
    xlab("Columns") +
    ylab("Values")
print(gg)

aov.out   = aov(val ~ grp, df)
anova.out = anova(aov.out)
print(anova.out)


## 
# Calc variance within group
nGrp        = length(unique(data.grp)) # different groups
var.within  = vector(mode = "numeric", length = nGrp)
mean.within = vector(mode = "numeric", length = nGrp)
for ( i in 1:nGrp ){
    idx              = data.grp == i
    var.within[[i]]  = var(data.val[idx])
    mean.within[[i]] = mean(data.val[idx])
}
# The average within group variance is equal to the "Residuals" Mean Sq in the anova table:
# This is Var_within = E[ Var[grp1], Var[grp2], ..., Var[grp5] ]
var.withinAverage = mean(var.within)
print(paste0("var.within = ", var.withinAverage))

# The variance between groups is given by
# Var_between = Var[ E[grp1],  E[grp2], ... E[grp5] ]
var.between = var(mean.within)
print(paste0("var.between = ", var.between))

##
# To obtain the MS_between term, we have to multiply by the sample size per group:
kSample    = sum(data.grp == 1) # number of elements within each group
MS.between = kSample * var.between
print(paste0("MS.between = kSample * var.between = ", MS.between))

##
# Thus, if we invert the last expression we get:
# var.between = MS.between / kSample

Which yields the following output:

Response: val
          Df Sum Sq Mean Sq F value   Pr(>F)   
grp        4 53.723 13.4307  6.0488 0.002332 **
Residuals 20 44.408  2.2204                    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
[1] "var.within = 2.2203876922"
[1] "var.between = 2.6861421478"
[1] "MS.between = kSample * var.between = 13.430710739"

The example above demonstrates

1. that the anova table is constructed by decomposing the sum-of-squares (and not the variance) into two components,
2. that the different elements in the anova table are not the within and between group variance components, and
3. how the anova table is constructed and how it is related to the within and between group variance.

Nevertheless, it is possible to decompose the total variance into a within group variance component and a between group variance component. However, these terms are defined differently. If you like to learn more about this decomposition you will have to read about variance components.

Answer 2

Using R as a calculator, the variance of the 25 observations (without regard to groups) is $4.088775.$

x = c(1.5377,    0.6923,    1.6501,    3.7950,    5.6715, 
      2.8339,    1.5664,    6.0349,    3.8759,    3.7925,
     -1.2588,    2.3426,    3.7254,    5.4897,    5.7172,
      1.8622,    5.5784,    2.9369,    5.4090,    6.6302,
      1.3188,    4.7694,    3.7147,    5.4172,    5.4889)
var(x)
[1] 4.088775

Also, SS(Total)/DF(Total) $= 4.088792,$ which is the same as above, except for slight rounding of SS(Total) = SS(Group) + SS(Resid) in your table.

53.723 + 44.408
[1] 98.131
98.131/24
[1] 4.088792