Splitting of treatment sum of squares in ANOVA

Question

An apparently classic mantra appearing in about a dozen of ANOVA textbooxs i consulted reads roughly as: given any choice of $T-1$ orthogonal contrasts $C_1, \cdots, C_{T-1}$ in a (not necessarily balanced) $T$-groups (one-way) ANOVA design, the treatment (or "between-groups") sum of squares $SS_{Treat}$ can be decomposed as the sum $\sum_{j=1}^{T-1}SS_{C_j}$ of contrasts $C_j$ sums of squares.

I think it would be helpful to have a proof of this fact or a reference to one, as most textbooks seem to lack.

I do not know if of any help, but here are some of the definitions of the statistics involved: let $\overline{Y}_i$ be the mean of the $n_i$ observations in the $i$-th group and $\overline{Y}$ the mean of all the observations; then

--a contrast is a linear combination $\sum_{i=1}^{T} \lambda_i\overline{Y}_i$ such that $\lambda_i\in \mathbb{R}$ and $\sum_i\lambda_i=0$;

--if two contrasts have coefficients $\lambda_i$ and $\nu_i$ such that $\sum_i\lambda_i\nu_i/n_i=0$, they are said to be orthogonal;

--sum of squares are defined as: $SS_{Treat}=\sum_{k=1}^Tn_k(\overline{Y}_k-\overline{Y})^2$ and $SS_C=\dfrac{C^2}{\sum_{i=1}^T\frac{\lambda_i^2}{n_i}}$

thanks a lot.

Answer 1

We let $C$ denote a contrast such as $C=\sum_{i=1}^T\lambda_i\bar{Y}_i$ and the vector $C=(\lambda_1,\cdots,\lambda_t)$. Define $C_0=\sum_{i=1}^Tn_i\bar{Y}_i$. Therefore, $C=\{C_0,C_1,\cdots,C_{T-1}\}$ forms an orthogonal basis for $\mathrm{R}^T$, with respect to the inner product $$\langle C_a,C_b\rangle=\sum_{i=1}^T\frac{\lambda_i^a\lambda_i^b}{n_i}.$$

Then, by the Parseval's identity, for any vector $M\in\mathrm{R}^T$, $$\|M\|^2 = \sum_{i=0}^{T-1}\frac{\langle C_i,M\rangle^2}{\langle C_i,C_i\rangle}.$$ Define $M=(\sqrt{n_1}\bar{Y}_1,\cdots,\sqrt{n_T}\bar{Y}_T)$, and check that for this specific $M$ we have, $$SS_{C_i}=\frac{\langle C_i,M\rangle^2}{\langle C_i,C_i\rangle}\;\;\;\text{for }i =0,\cdots,T.$$ Therefore, since $\|M\|^2=\sum_{i=1}^Tn_i\bar{Y}_i^2$, $$\sum n_i\bar{Y}_i^2=N\bar{Y}^2+\sum_{i=1}^{T-1}SS_{C_i},$$ where $N=n_1+\cdots+n_T$ and we let $SS_{C_0}=N\bar{Y}^2$.Therefore, $$\sum_{i=1}^{T-1}SS_{C_i}=\sum_{i=1}^Tn_i\bar{Y}^2_i-N\bar{Y}^2=\sum_{i=1}^Tn_i(\bar{Y}-\bar{Y}_i)^2=SS_{treat}.$$

QED

---

Just 3 comments:

• The fact that $C$ forms a basis follows from the fact that coordinates of $C_1,\cdots,C_{T-1}$ add up to zero. So every linear combination of them has this property. But $C_0$ does not have this property. Therefore $C_0$ is orthogonal to $C_1,\cdots,C_{T-1}$. On the other hand, every $T$ orthogonal vectors form a basis for $\mathrm{R}^T$.
• The Parseval's identity states that for orthonormal basis $\{v_i\}$, we have $$\|M\|^2=\sum\langle v_i,M\rangle.$$ To apply this to the orthogonal basis $C$, you can first divide the vector to their length to get the orthonormal basis $\{\frac{C_0}{\sqrt{\langle C_0,C_0\rangle}},\cdots,\frac{C_{T-1}}{\sqrt{\langle C_{T-1},C_{T-1}\rangle}}\}$
• note that $\|M\|^2$ is not the Euclidean length. It is the length defined by the inner product.

Answer 2

@pmjn6's answer is not based on standard notation so it can be a bit confusing. Based on his answer though, the Parseval's identity is the key to decomposing the sums of squares. Constructively, take the contrasts $$c_l = \frac{1}{\sqrt{l(l+1)}}(\underbrace{1, 1, \ldots, 1}_{l\text{ times}}, -l, 0, \ldots, 0)^\prime,\quad l=1,\ldots,a-1.$$ Now, take $c_0 = 1_a/\sqrt{a}$ where $1_a$ is a vector of ones. $\{c_0,c_1,\ldots,c_a\}$ forms an orthonormal basis for $\mathbb{R}^a$. By Parseval's identity, for any vector $v \in \mathbb{R}^a$, $$\|v\|^2 = \sum_{i=0}^a (c_i^\prime v)^2.$$ Therefore, take $v = \sqrt{n}(\bar{Y}_{1\cdot},\ldots,\bar{Y}_{a\cdot})^\prime$. We have $$\mathrm{SS}_{c_i} = (c_i^\prime v)^2,\qquad i=0,\ldots,a$$, thus, $\|v\|^2 = \sum_{i=1}^a n\bar{Y}_{i\cdot}^2$, and $$\sum n\bar{Y}_{i\cdot}^2 = na\bar{Y}_{\cdot \cdot}^2 + \sum_{i=1}^a \mathrm{SS}_{c_i},$$ where $\mathrm{SS}_{c_0} = na\bar{Y}_{\cdot \cdot}^2$. Thus, $$\sum_{i=1}^a \mathrm{SS}_{c_i} = \sum_{i=1}^a n\bar{Y}_{i\cdot}^2 - na\bar{Y}_{\cdot\cdot}^2 = \sum_{i=1}^a n(\bar{Y}_{\cdot\cdot}-\bar{Y}_{i\cdot})^2.$$ The sums of squares due to the contrasts decompose the treatment sum of squares, or $\mathrm{SSR}_c$.