An apparently classic mantra appearing in about a dozen of ANOVA textbooxs i consulted reads roughly as: given any choice of $T-1$ orthogonal contrasts $C_1, \cdots, C_{T-1}$ in a (not necessarily balanced) $T$-groups (one-way) ANOVA design, the treatment (or "between-groups") sum of squares $SS_{Treat}$ can be decomposed as the sum $\sum_{j=1}^{T-1}SS_{C_j}$ of contrasts $C_j$ sums of squares.

I think it would be helpful to have a proof of this fact or a reference to one, as most textbooks seem to lack.

I do not know if of any help, but here are some of the definitions of the statistics involved: let $\overline{Y}_i$ be the mean of the $n_i$ observations in the $i$-th group and $\overline{Y}$ the mean of all the observations; then

--a contrast is a linear combination $\sum_{i=1}^{T} \lambda_i\overline{Y}_i$ such that $\lambda_i\in \mathbb{R}$ and $\sum_i\lambda_i=0$;

--if two contrasts have coefficients $\lambda_i$ and $\nu_i$ such that $\sum_i\lambda_i\nu_i/n_i=0$, they are said to be orthogonal;

--sum of squares are defined as: $SS_{Treat}=\sum_{k=1}^Tn_k(\overline{Y}_k-\overline{Y})^2$ and $SS_C=\dfrac{C^2}{\sum_{i=1}^T\frac{\lambda_i^2}{n_i}}$

thanks a lot.


2 Answers 2


We let $C$ denote a contrast such as $C=\sum_{i=1}^T\lambda_i\bar{Y}_i$ and the vector $C=(\lambda_1,\cdots,\lambda_t)$. Define $C_0=\sum_{i=1}^Tn_i\bar{Y}_i$. Therefore, $C=\{C_0,C_1,\cdots,C_{T-1}\}$ forms an orthogonal basis for $\mathrm{R}^T$, with respect to the inner product $$\langle C_a,C_b\rangle=\sum_{i=1}^T\frac{\lambda_i^a\lambda_i^b}{n_i}.$$

Then, by the Parseval's identity, for any vector $M\in\mathrm{R}^T$, $$\|M\|^2 = \sum_{i=0}^{T-1}\frac{\langle C_i,M\rangle^2}{\langle C_i,C_i\rangle}.$$ Define $M=(\sqrt{n_1}\bar{Y}_1,\cdots,\sqrt{n_T}\bar{Y}_T)$, and check that for this specific $M$ we have, $$SS_{C_i}=\frac{\langle C_i,M\rangle^2}{\langle C_i,C_i\rangle}\;\;\;\text{for }i =0,\cdots,T.$$ Therefore, since $\|M\|^2=\sum_{i=1}^Tn_i\bar{Y}_i^2$, $$\sum n_i\bar{Y}_i^2=N\bar{Y}^2+\sum_{i=1}^{T-1}SS_{C_i},$$ where $N=n_1+\cdots+n_T$ and we let $SS_{C_0}=N\bar{Y}^2$.Therefore, $$\sum_{i=1}^{T-1}SS_{C_i}=\sum_{i=1}^Tn_i\bar{Y}^2_i-N\bar{Y}^2=\sum_{i=1}^Tn_i(\bar{Y}-\bar{Y}_i)^2=SS_{treat}.$$


Just 3 comments:

  • The fact that $C$ forms a basis follows from the fact that coordinates of $C_1,\cdots,C_{T-1}$ add up to zero. So every linear combination of them has this property. But $C_0$ does not have this property. Therefore $C_0$ is orthogonal to $C_1,\cdots,C_{T-1}$. On the other hand, every $T$ orthogonal vectors form a basis for $\mathrm{R}^T$.
  • The Parseval's identity states that for orthonormal basis $\{v_i\}$, we have $$\|M\|^2=\sum\langle v_i,M\rangle.$$ To apply this to the orthogonal basis $C$, you can first divide the vector to their length to get the orthonormal basis $\{\frac{C_0}{\sqrt{\langle C_0,C_0\rangle}},\cdots,\frac{C_{T-1}}{\sqrt{\langle C_{T-1},C_{T-1}\rangle}}\}$
  • note that $\|M\|^2$ is not the Euclidean length. It is the length defined by the inner product.

@pmjn6's answer is not based on standard notation so it can be a bit confusing. Based on his answer though, the Parseval's identity is the key to decomposing the sums of squares. Constructively, take the contrasts $$c_l = \frac{1}{\sqrt{l(l+1)}}(\underbrace{1, 1, \ldots, 1}_{l\text{ times}}, -l, 0, \ldots, 0)^\prime,\quad l=1,\ldots,a-1.$$ Now, take $c_0 = 1_a/\sqrt{a}$ where $1_a$ is a vector of ones. $\{c_0,c_1,\ldots,c_a\}$ forms an orthonormal basis for $\mathbb{R}^a$. By Parseval's identity, for any vector $v \in \mathbb{R}^a$, $$\|v\|^2 = \sum_{i=0}^a (c_i^\prime v)^2.$$ Therefore, take $v = \sqrt{n}(\bar{Y}_{1\cdot},\ldots,\bar{Y}_{a\cdot})^\prime$. We have $$\mathrm{SS}_{c_i} = (c_i^\prime v)^2,\qquad i=0,\ldots,a$$, thus, $\|v\|^2 = \sum_{i=1}^a n\bar{Y}_{i\cdot}^2$, and $$\sum n\bar{Y}_{i\cdot}^2 = na\bar{Y}_{\cdot \cdot}^2 + \sum_{i=1}^a \mathrm{SS}_{c_i},$$ where $\mathrm{SS}_{c_0} = na\bar{Y}_{\cdot \cdot}^2$. Thus, $$\sum_{i=1}^a \mathrm{SS}_{c_i} = \sum_{i=1}^a n\bar{Y}_{i\cdot}^2 - na\bar{Y}_{\cdot\cdot}^2 = \sum_{i=1}^a n(\bar{Y}_{\cdot\cdot}-\bar{Y}_{i\cdot})^2.$$ The sums of squares due to the contrasts decompose the treatment sum of squares, or $\mathrm{SSR}_c$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.