3
$\begingroup$

I was trying to understand orthogonal contrasts in ANOVA following the article by Harry M. Schey "A Geometric Description of Orthogonal Contrasts in One-Way Analysis of Variance" in The American Statistician, May 1985.

The issue has to do with the description in the paper of these vectors, $\Phi_0$, defined as $[(n_1/n)^{1/2}\,(n_2/n)^{1/2} \,...(n_r/n)^{1/2}]'$, in which every $n_i$ stands for the number of observations in one factor (treatment), and $r$ is the overall number of factors.

$ W$ is defined as $I_r - \Phi_0\Phi_0'$ with $I_r$ representing the identity matrix with $r$ rows.

The statement that I have problems with is that the $SS_T$ or sum of squares between groups, defined as $\sum_{i} n_i(\bar{y}_i -\bar{y})^2$ with $\bar{y}$ representing the overall mean, can be alternatively be expressed as $z'Wz$ with $z = [n_1^{1/2}\bar{y}_1\,\,n_2^{1/2}\bar{y}_2\,\,...n_r^{1/2}\bar{y}_r]'$ according to the author.

I went through the linear algebra, and I probably made a mistake that I can't detect, but the $z'Wz$ ends up as $\bar{y}_1n_1(\bar{y}_1\,-\bar{y})\,+\,\bar{y}_2\,n_2\,(\bar{y}_2-\bar{y})\,...\,\bar{y}_rn_r\,(\bar{y}_r-\bar{y})$. Not entirely dissimilar to the original definition, but lacking the square, and with an added $\bar{y}_i$.

What am I missing?

$\endgroup$
3
$\begingroup$

Looks like $W$ is idempotent, hence $zWz' = zWWz'$, using your notation. That will give you the correct expression. Just guessing, I did not check.

$\endgroup$
  • 1
    $\begingroup$ Yes. W is idempotent - I was able to prove this to myself after reading it on the manuscript I'm referencing. And I see that this is key to solving my question. I just need to see the dance of rows and columns coming together... $\endgroup$ – Antoni Parellada Apr 9 '15 at 13:52
2
$\begingroup$

... And after going over the linear algebra a bunch of times, and incorporating the idempotent property as mentioned in the article, and reminded by F. Tusell - thank you! - I got it to work... So just in case someone out there gets stuck with the same "trick" of using idempotent matrices to solve sum of squares, here it goes:

First off, $W$ is the result of:

$\small W = I_r - \Phi_0 \Phi_0'= \begin{bmatrix} 1&0&0&...&0_r\\ 0&1&0&...&0_r\\ 0&0&1&...&0_r\\ 0&0&0&...&1_r \end{bmatrix} -\begin{bmatrix} \sqrt{\frac{n_1}{n}}\\ \sqrt{\frac{n_2}{n}}\\ ...\\ \sqrt{\frac{n_r}{n}}\\ \end{bmatrix} \begin{bmatrix} \sqrt{\frac{n_1}{n}}& \sqrt{\frac{n_2}{n}}& ...& \sqrt{\frac{n_r}{n}}\\ \end{bmatrix}=$

$\small=\begin{bmatrix} 1&0&0&...&0_r\\ 0&1&0&...&0_r\\ 0&0&1&...&0_r\\ 0&0&0&...&1_r \end{bmatrix} -\begin{bmatrix} \sqrt{\frac{n_1}{n}}\sqrt{\frac{n_1}{n}}&\sqrt{\frac{n_1}{n}}\sqrt{\frac{n_2}{n}}&...&\sqrt{\frac{n_1}{n}}\sqrt{\frac{n_r}{n}} \\ \sqrt{\frac{n_2}{n}}\sqrt{\frac{n_1}{n}}&\sqrt{\frac{n_2}{n}}\sqrt{\frac{n_2}{n}}&...&\sqrt{\frac{n_2}{n}}\sqrt{\frac{n_r}{n}}\\ ...&...&...&...\\ \sqrt{\frac{n_r}{n}}\sqrt{\frac{n_1}{n}}&\sqrt{\frac{n_r}{n}}\sqrt{\frac{n_2}{n}}&...&\sqrt{\frac{n_r}{n}}\sqrt{\frac{n_r}{n}}\\ \end{bmatrix}=$

$\small=\frac{1}{n}\begin{bmatrix} n -\sqrt{n_1}\sqrt{n_1}&-\sqrt{n_1}\sqrt{n_2}&...&-\sqrt{n_1}\sqrt{n_r} \\ \\-\sqrt{n_2}\sqrt{n_1}&n-\sqrt{n_2}\sqrt{n_2}&...&-\sqrt{n_2}\sqrt{n_r}\\ ...&...&...&...\\ -\sqrt{n_r}\sqrt{n_1}&-\sqrt{n_r}\sqrt{n_2}&...&n-\sqrt{n_r}\sqrt{n_r}\\ \end{bmatrix}$

This can be proven to be symmetrical and idempotent - just taking the first dot product of $WW$...

$\small\begin{bmatrix}1- \frac{n_1}{n}&-\sqrt{\frac{n_1}{n}}\sqrt{\frac{n_2}{n}}&...&-\sqrt{\frac{n_1}{n}}\sqrt{\frac{n_r}{n}}\end{bmatrix}\begin{bmatrix}1- \frac{n_1}{n}&-\sqrt{\frac{n_1}{n}}\sqrt{\frac{n_2}{n}}&...&-\sqrt{\frac{n_1}{n}}\sqrt{\frac{n_r}{n}}\end{bmatrix}^{T}=\\ 1- 2\frac{n_1}{n}+\frac{n_1^2}{n^2} + \frac{n_1 n_2}{n^2} +...+ \frac{n_1 n_r}{n^2}= 1 - 2\frac{n_1}{n} +\frac{n_1}{n}(\frac{n_1}{n}+\frac{n_2}{n}+...+\frac{n_r}{n}) = 1 - 2\frac{n_1}{n} +\frac{n_1}{n}(\frac{n}{n})=1 - \frac{n_1}{n}$.

And justifying,

$SS_{treatment\, =\, between\, groups} = z'Wz = z'WWz$

On the other hand,

$\small z = \begin{bmatrix} \sqrt{n_1}&\bar{y}_1\\ \sqrt{n_2}&\bar{y}_2\\ ...\\ \sqrt{n_r}&\bar{y}_r\\ \end{bmatrix} $

Therefore,

$z'WWz=$

$\small\begin{bmatrix} \sqrt{n_1}\bar{y}_1\\ \sqrt{n_2}\bar{y}_2\\ ...\\ \sqrt{n_r}\bar{y}_r \end{bmatrix}^{T} \small\frac{1}{n^2}\begin{bmatrix} n -\sqrt{n_1}\sqrt{n_1}&-\sqrt{n_1}\sqrt{n_2}&...&-\sqrt{n_1}\sqrt{n_r} \\ \\-\sqrt{n_2}\sqrt{n_1}&n-\sqrt{n_2}\sqrt{n_2}&...&-\sqrt{n_2}\sqrt{n_r}\\ ...&...&...&...\\ -\sqrt{n_r}\sqrt{n_1}&-\sqrt{n_r}\sqrt{n_2}&...&n-\sqrt{n_r}\sqrt{n_r}\\ \end{bmatrix}^2\begin{bmatrix} \sqrt{n_1}&\bar{y}_1\\ \sqrt{n_2}&\bar{y}_2\\ ...\\ \sqrt{n_r}&\bar{y}_r\\ \end{bmatrix}$

Using the associative property, and separating the two apposed $W$ matrices:

$\small\frac{1}{n}\begin{bmatrix} \sqrt{n_1}(n\,\bar{y}_1\,-[n_1 \bar{y}_1\, +n_2\bar{y}_2+...+n_r\bar{y}_r])\\ \sqrt{n_2}(n\,\bar{y}_2\,-[n_1 \bar{y}_1\, +n_2\bar{y}_2+...+n_r\bar{y}_r])\\...\\ \sqrt{n_r}(n\bar{y}_r\,-[n_1 \bar{y}_1\, +n_2\bar{y}_2+...+n_r\bar{y}_r]) \end{bmatrix}^{T}\frac{1}{n}\begin{bmatrix} \sqrt{n_1}(n\,\bar{y}_1\,-[n_1 \bar{y}_1\, +n_2\bar{y}_2+...+n_r\bar{y}_r])\\ \sqrt{n_2}(n\,\bar{y}_2\,-[n_1 \bar{y}_1\, +n_2\bar{y}_2+...+n_r\bar{y}_r])\\...\\ \sqrt{n_r}(n\bar{y}_r\,-[n_1 \bar{y}_1\, +n_2\bar{y}_2+...+n_r\bar{y}_r]) \end{bmatrix}=$

$\small \begin{bmatrix}\sqrt{n_1}(\bar{y}_1-\bar{y})&\sqrt{n_2}(\bar{y}_2-\bar{y}),...,\sqrt{n_r}(\bar{y}_r-\bar{y}) \end{bmatrix}\begin{bmatrix}\sqrt{n_1}(\bar{y}_1-\bar{y})\\\sqrt{n_2}(\bar{y}_2-\bar{y})\\\vdots\\\sqrt{n_r}(\bar{y}_r-\bar{y}) \end{bmatrix} = \sum_{i}n_i(\bar{y}_i-\bar{y})^2$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.