ANOVA: What is estimated by the Mean Squares?

Question

One step in running an ANOVA is calculating the mean sum of squares (MS) for each term (or source of variation) in the model. I have been trying to get my head around what these MS are an estimate of. In other words, what property of the population are MS estimating and how do they relate to each other?

For example, consider a simple one-way ANOVA with A as a fixed factor. One would calculate:

1. $MS_{\text{between}}$
2. $MS_{\text{within}}$
3. $MS_{\text{total}}$

Now, some texts/books I have consulted, state that the MS_between is a 'measure of the variance including both error and factor effects'. One book gives the following formulas $$ MS_\text{between} = (n \times \sigma^2_\text{between}) + \sigma^2_\text{within} $$ and $$ MS_\text{within} = \sigma^2_\text{within} $$

However, this would imply that $MS_\text{between}$ should always be a equal or larger than the $MS_\text{within}$. This is certainly not true. For example, the mean for each level of A could be equal which would leave $MS_\text{between}=0$.

But also, conceptually speaking, would the first equation not imply that $MS_\text{between} = MS_\text{total}$? After all, it seems to estimate total variance due to both random variation and effect A. And isn't this what MS_total is already estimating?

Questions:

1. If $MS_{\text{between}}$ is a 'measure of the variance including both error and factor effects' why is it different from the $MS_{\text{total}}$?
2. And why can $MS_{\text{between}}$ be smaller than $MS_{\text{within}}$ (given that $MS_{\text{within is a part of $MS_{\text{between}}$, as per the first formula above)

Any clarification would be greatly appreciated.

Answer 1

Answer to Q1

MS_total and MS_between estimate population quantities that include both the error ($\varepsilon$) and the factor ($\beta$) effects, but the error and factor effects are combined in different ways.

MS_total estimates $\frac{an - n}{an - 1}Var(\beta) + Var(\varepsilon)$

MS_between estimates $nVar(\beta) + Var(\varepsilon)$

MS_within estimates $Var(\varepsilon)$

MS_total and MS_between therefore estimate different population quantities.

The population quantity that MS_total estimates must always be less than the population quantity that MS_between estimates. This is because an-n is always less than an-1 so MS_total will only include a a fraction of Var($\beta$) whereas MS_between will include n times Var($\beta$).

Answer to Q2

The population quantity that MS_between estimates must always be greater than or equal to the population quantity that MS_within estimates (under assumptions of independence).

The value of MS_between calculated from sample data in the ANOVA is sometimes less than MS_error calculated from sample data due to random sampling error.

Further Comments

The question considers a “one way ANOVA with A as a fixed factor” but then gives estimated variance components for a random factor model. The answers above are for a random factor model with the same number of samples in every group, but the same principle holds for the fixed factor model.

The question also treats the calculated values of Mean Squares from the ANOVA as being the same as the population quantities that are estimated by the Mean Squares. Manipulating the group means to obtain a calculated MS_between less than the calculated MS_error does not provide a contradiction to the equations above.

I have not come across any recommendations to calculate and use the MS_total in any of the statistical texts that I use, and none of them considered what, if anything, was estimated by MS_total. I derived this myself in the following way:

$MS_{total} = \frac{SS_{total}}{an-1} = \frac{SS_{within}}{an-1} + \frac{SS_{between}}{an-1} = \frac{a(n-1)}{an-1}MS_{within} + \frac{a-1}{an-1}MS_{between}$

which means that

MS_total estimates $\frac{a(n-1)}{an-1}Var(\varepsilon) + \frac{a-1}{an-1}nVar(\beta) + \frac{a-1}{an-1}Var(\varepsilon)$

which gives

MS_total estimates $\frac{an - n}{an - 1}Var(\beta) + Var(\varepsilon)$