One step in running an ANOVA is calculating the mean sum of squares (MS) for each term (or source of variation) in the model. I have been trying to get my head around what these MS are an estimate of. In other words, what property of the population are MS estimating and how do they relate to each other?

For example, consider a simple one-way ANOVA with A as a fixed factor. One would calculate:

  1. $MS_{\text{between}}$
  2. $MS_{\text{within}}$
  3. $MS_{\text{total}}$

Now, some texts/books I have consulted, state that the MS_between is a 'measure of the variance including both error and factor effects'. One book gives the following formulas $$ MS_\text{between} = (n \times \sigma^2_\text{between}) + \sigma^2_\text{within} $$ and $$ MS_\text{within} = \sigma^2_\text{within} $$

However, this would imply that $MS_\text{between}$ should always be a equal or larger than the $MS_\text{within}$. This is certainly not true. For example, the mean for each level of A could be equal which would leave $MS_\text{between}=0$.

But also, conceptually speaking, would the first equation not imply that $MS_\text{between} = MS_\text{total}$? After all, it seems to estimate total variance due to both random variation and effect A. And isn't this what MS_total is already estimating?


  1. If $MS_{\text{between}}$ is a 'measure of the variance including both error and factor effects' why is it different from the $MS_{\text{total}}$?
  2. And why can $MS_{\text{between}}$ be smaller than $MS_{\text{within}}$ (given that $MS_{\text{within is a part of $MS_{\text{between}}$, as per the first formula above)

Any clarification would be greatly appreciated.

  • 1
    $\begingroup$ Can you clarify whether your question relates to Mean Squares (MS), or Sums of Squares (SS). They are not the same and there is inconsistency in the question (for example, Q1 asks about SS_Between and MS_Total, and you would not calculate MS_Total but you would calculate SS_Total). $\endgroup$ Aug 13, 2016 at 12:19
  • $\begingroup$ Thank you for pointing out the inconsistencies in my question. I've updated the text to pertain mean sum of squares only. $\endgroup$ Aug 13, 2016 at 14:39
  • $\begingroup$ The calculation of MS_Total is not something I've ever come across. That does not make sense to me. Calculation of SS_Total, on the other hand, is essential for ANOVA. Can you cite a reference that explains how to calculate MS_Total and why it is necessary? $\endgroup$ Aug 13, 2016 at 14:46
  • 1
    $\begingroup$ You are right that MS_total is not something that is usually found in anova tables. However, for the purpose of this question I assumed that I can be calculated from SS_total by division by the degrees of freedom (n-1) for SS_total. In fact, $SS_{Total} / n-1$ would actually be an unbiased estimate of the population's variance ($\sigma^2_{population}$). $\endgroup$ Aug 13, 2016 at 14:56
  • $\begingroup$ [continued] However, even if you don't agree with this, there is still my second question. This one does not depend on MS_total (or its definition). Would you have any comment on this second question? $\endgroup$ Aug 13, 2016 at 15:02

1 Answer 1


Answer to Q1

MS_total and MS_between estimate population quantities that include both the error ($\varepsilon$) and the factor ($\beta$) effects, but the error and factor effects are combined in different ways.

MS_total estimates $\frac{an - n}{an - 1}Var(\beta) + Var(\varepsilon)$

MS_between estimates $nVar(\beta) + Var(\varepsilon)$

MS_within estimates $Var(\varepsilon)$

MS_total and MS_between therefore estimate different population quantities.

The population quantity that MS_total estimates must always be less than the population quantity that MS_between estimates. This is because an-n is always less than an-1 so MS_total will only include a a fraction of Var($\beta$) whereas MS_between will include n times Var($\beta$).

Answer to Q2

The population quantity that MS_between estimates must always be greater than or equal to the population quantity that MS_within estimates (under assumptions of independence).

The value of MS_between calculated from sample data in the ANOVA is sometimes less than MS_error calculated from sample data due to random sampling error.

Further Comments

The question considers a “one way ANOVA with A as a fixed factor” but then gives estimated variance components for a random factor model. The answers above are for a random factor model with the same number of samples in every group, but the same principle holds for the fixed factor model.

The question also treats the calculated values of Mean Squares from the ANOVA as being the same as the population quantities that are estimated by the Mean Squares. Manipulating the group means to obtain a calculated MS_between less than the calculated MS_error does not provide a contradiction to the equations above.

I have not come across any recommendations to calculate and use the MS_total in any of the statistical texts that I use, and none of them considered what, if anything, was estimated by MS_total. I derived this myself in the following way:

$MS_{total} = \frac{SS_{total}}{an-1} = \frac{SS_{within}}{an-1} + \frac{SS_{between}}{an-1} = \frac{a(n-1)}{an-1}MS_{within} + \frac{a-1}{an-1}MS_{between}$

which means that

MS_total estimates $\frac{a(n-1)}{an-1}Var(\varepsilon) + \frac{a-1}{an-1}nVar(\beta) + \frac{a-1}{an-1}Var(\varepsilon)$

which gives

MS_total estimates $\frac{an - n}{an - 1}Var(\beta) + Var(\varepsilon)$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.