Geometric Interpretation of Orthogonal Contrasts in ANOVA

Question

I was trying to understand orthogonal contrasts in ANOVA following the article by Harry M. Schey "A Geometric Description of Orthogonal Contrasts in One-Way Analysis of Variance" in The American Statistician, May 1985.

The issue has to do with the description in the paper of these vectors, $\Phi_0$, defined as $[(n_1/n)^{1/2}\,(n_2/n)^{1/2} \,...(n_r/n)^{1/2}]'$, in which every $n_i$ stands for the number of observations in one factor (treatment), and $r$ is the overall number of factors.

$ W$ is defined as $I_r - \Phi_0\Phi_0'$ with $I_r$ representing the identity matrix with $r$ rows.

The statement that I have problems with is that the $SS_T$ or sum of squares between groups, defined as $\sum_{i} n_i(\bar{y}_i -\bar{y})^2$ with $\bar{y}$ representing the overall mean, can be alternatively be expressed as $z'Wz$ with $z = [n_1^{1/2}\bar{y}_1\,\,n_2^{1/2}\bar{y}_2\,\,...n_r^{1/2}\bar{y}_r]'$ according to the author.

I went through the linear algebra, and I probably made a mistake that I can't detect, but the $z'Wz$ ends up as $\bar{y}_1n_1(\bar{y}_1\,-\bar{y})\,+\,\bar{y}_2\,n_2\,(\bar{y}_2-\bar{y})\,...\,\bar{y}_rn_r\,(\bar{y}_r-\bar{y})$. Not entirely dissimilar to the original definition, but lacking the square, and with an added $\bar{y}_i$.

What am I missing?

Answer 1

Looks like $W$ is idempotent, hence $zWz' = zWWz'$, using your notation. That will give you the correct expression. Just guessing, I did not check.

Answer 2

... And after going over the linear algebra a bunch of times, and incorporating the idempotent property as mentioned in the article, and reminded by F. Tusell - thank you! - I got it to work... So just in case someone out there gets stuck with the same "trick" of using idempotent matrices to solve sum of squares, here it goes:

First off, $W$ is the result of:

$\small W = I_r - \Phi_0 \Phi_0'= \begin{bmatrix} 1&0&0&...&0_r\\ 0&1&0&...&0_r\\ 0&0&1&...&0_r\\ 0&0&0&...&1_r \end{bmatrix} -\begin{bmatrix} \sqrt{\frac{n_1}{n}}\\ \sqrt{\frac{n_2}{n}}\\ ...\\ \sqrt{\frac{n_r}{n}}\\ \end{bmatrix} \begin{bmatrix} \sqrt{\frac{n_1}{n}}& \sqrt{\frac{n_2}{n}}& ...& \sqrt{\frac{n_r}{n}}\\ \end{bmatrix}=$

$\small=\begin{bmatrix} 1&0&0&...&0_r\\ 0&1&0&...&0_r\\ 0&0&1&...&0_r\\ 0&0&0&...&1_r \end{bmatrix} -\begin{bmatrix} \sqrt{\frac{n_1}{n}}\sqrt{\frac{n_1}{n}}&\sqrt{\frac{n_1}{n}}\sqrt{\frac{n_2}{n}}&...&\sqrt{\frac{n_1}{n}}\sqrt{\frac{n_r}{n}} \\ \sqrt{\frac{n_2}{n}}\sqrt{\frac{n_1}{n}}&\sqrt{\frac{n_2}{n}}\sqrt{\frac{n_2}{n}}&...&\sqrt{\frac{n_2}{n}}\sqrt{\frac{n_r}{n}}\\ ...&...&...&...\\ \sqrt{\frac{n_r}{n}}\sqrt{\frac{n_1}{n}}&\sqrt{\frac{n_r}{n}}\sqrt{\frac{n_2}{n}}&...&\sqrt{\frac{n_r}{n}}\sqrt{\frac{n_r}{n}}\\ \end{bmatrix}=$

$\small=\frac{1}{n}\begin{bmatrix} n -\sqrt{n_1}\sqrt{n_1}&-\sqrt{n_1}\sqrt{n_2}&...&-\sqrt{n_1}\sqrt{n_r} \\ \\-\sqrt{n_2}\sqrt{n_1}&n-\sqrt{n_2}\sqrt{n_2}&...&-\sqrt{n_2}\sqrt{n_r}\\ ...&...&...&...\\ -\sqrt{n_r}\sqrt{n_1}&-\sqrt{n_r}\sqrt{n_2}&...&n-\sqrt{n_r}\sqrt{n_r}\\ \end{bmatrix}$

This can be proven to be symmetrical and idempotent - just taking the first dot product of $WW$...

$\small\begin{bmatrix}1- \frac{n_1}{n}&-\sqrt{\frac{n_1}{n}}\sqrt{\frac{n_2}{n}}&...&-\sqrt{\frac{n_1}{n}}\sqrt{\frac{n_r}{n}}\end{bmatrix}\begin{bmatrix}1- \frac{n_1}{n}&-\sqrt{\frac{n_1}{n}}\sqrt{\frac{n_2}{n}}&...&-\sqrt{\frac{n_1}{n}}\sqrt{\frac{n_r}{n}}\end{bmatrix}^{T}=\\ 1- 2\frac{n_1}{n}+\frac{n_1^2}{n^2} + \frac{n_1 n_2}{n^2} +...+ \frac{n_1 n_r}{n^2}= 1 - 2\frac{n_1}{n} +\frac{n_1}{n}(\frac{n_1}{n}+\frac{n_2}{n}+...+\frac{n_r}{n}) = 1 - 2\frac{n_1}{n} +\frac{n_1}{n}(\frac{n}{n})=1 - \frac{n_1}{n}$.

And justifying,

$SS_{treatment\, =\, between\, groups} = z'Wz = z'WWz$

On the other hand,

$\small z = \begin{bmatrix} \sqrt{n_1}&\bar{y}_1\\ \sqrt{n_2}&\bar{y}_2\\ ...\\ \sqrt{n_r}&\bar{y}_r\\ \end{bmatrix} $

Therefore,

$z'WWz=$

$\small\begin{bmatrix} \sqrt{n_1}\bar{y}_1\\ \sqrt{n_2}\bar{y}_2\\ ...\\ \sqrt{n_r}\bar{y}_r \end{bmatrix}^{T} \small\frac{1}{n^2}\begin{bmatrix} n -\sqrt{n_1}\sqrt{n_1}&-\sqrt{n_1}\sqrt{n_2}&...&-\sqrt{n_1}\sqrt{n_r} \\ \\-\sqrt{n_2}\sqrt{n_1}&n-\sqrt{n_2}\sqrt{n_2}&...&-\sqrt{n_2}\sqrt{n_r}\\ ...&...&...&...\\ -\sqrt{n_r}\sqrt{n_1}&-\sqrt{n_r}\sqrt{n_2}&...&n-\sqrt{n_r}\sqrt{n_r}\\ \end{bmatrix}^2\begin{bmatrix} \sqrt{n_1}&\bar{y}_1\\ \sqrt{n_2}&\bar{y}_2\\ ...\\ \sqrt{n_r}&\bar{y}_r\\ \end{bmatrix}$

Using the associative property, and separating the two apposed $W$ matrices:

$\small\frac{1}{n}\begin{bmatrix} \sqrt{n_1}(n\,\bar{y}_1\,-[n_1 \bar{y}_1\, +n_2\bar{y}_2+...+n_r\bar{y}_r])\\ \sqrt{n_2}(n\,\bar{y}_2\,-[n_1 \bar{y}_1\, +n_2\bar{y}_2+...+n_r\bar{y}_r])\\...\\ \sqrt{n_r}(n\bar{y}_r\,-[n_1 \bar{y}_1\, +n_2\bar{y}_2+...+n_r\bar{y}_r]) \end{bmatrix}^{T}\frac{1}{n}\begin{bmatrix} \sqrt{n_1}(n\,\bar{y}_1\,-[n_1 \bar{y}_1\, +n_2\bar{y}_2+...+n_r\bar{y}_r])\\ \sqrt{n_2}(n\,\bar{y}_2\,-[n_1 \bar{y}_1\, +n_2\bar{y}_2+...+n_r\bar{y}_r])\\...\\ \sqrt{n_r}(n\bar{y}_r\,-[n_1 \bar{y}_1\, +n_2\bar{y}_2+...+n_r\bar{y}_r]) \end{bmatrix}=$

$\small \begin{bmatrix}\sqrt{n_1}(\bar{y}_1-\bar{y})&\sqrt{n_2}(\bar{y}_2-\bar{y}),...,\sqrt{n_r}(\bar{y}_r-\bar{y}) \end{bmatrix}\begin{bmatrix}\sqrt{n_1}(\bar{y}_1-\bar{y})\\\sqrt{n_2}(\bar{y}_2-\bar{y})\\\vdots\\\sqrt{n_r}(\bar{y}_r-\bar{y}) \end{bmatrix} = \sum_{i}n_i(\bar{y}_i-\bar{y})^2$.