Why is maximum number of orthogonal contrasts equal to the number of groups minus one and what does mutually orthogonal mean?

Question

1. I was wondering why, for $g$ groups, it is always possible to construct $g - 1$ mutually orthogonal contrasts and $g - 1$ is the maximum number?

   A counterexample I have is that for two groups with means $\mu_1$ and $\mu_2$, $g=2$, and $\mu_1$ and $\mu_2$ themselves are two orthogonal contrasts.

2. The definition of orthogonal contrasts I have learned is a set of contrasts in which, for any distinct pair, the sum of the cross-products of the coefficients is zero. So it is a pairwise concept. What does "mutually orthogonal contrasts" mean, and is it not defined pairwise?

Thanks!

Answer 1

1. What are contrasts? Contrasts are linear combinations of treatment group means $\mu_i$ used to assess for specific differences between groups in an ANOVA (omnibus test) with a positive result. These linear combinations are of the form: $\Lambda = \sum_{i}c_i\mu_i$ with the condition $\sum_{i}c_i=0$. In these equations, $c_i$ represent the contrasts coefficients. Of note, these are scalar measures. Since we deal with a sample, the estimator of $\Lambda$ uses the group (or treatment) mean, $\bar{y}$. So a contrast between groups will be of the form $\hat\Lambda_j = \sum_{i}c_{ji}\bar{y}_i$. The index i referring to the groups involved in the contrast j. These contrasts are then tested using an F-test statistic.

2. What are mutually orthogonal contrasts? They are contrasts that satisfy $\sum_{i}\frac{c_{ji}c_{ki}}{n_i}=0$ with $j$ and $k$ indicating the contrasts indices, and $i$ the observation or subject ($n_i$ being the number of subjects in a group, out of a total of $r$ groups). Expressed in matrix form the crossproduct of the contrast matrix will have zeros in all off-diagonal entries.

Here is a made-up example contrasting the number of bacterial colonies grown (independent variable) in multiple observations for groups exposed to different antibiotics (penicillin ('pnc'), cloramphenicol ('cloram'), metronidazole ('metro'), or simply soap, or nothing ('control')).

The contrasts allowed will enable four (r - 1) reasonable questions to be answered, including: "Are antibiotics better at suppressing bacterial growth than soap (or nothing)?" (contrast 1: C1 <- c(2,-3,2,2,-3)); "Is penicillin superior to other antibiotics?" (contrast 2: C2 <- c(2,0,-1,-1,0)); "Does cloramphenicol differ from metronidazole?" (contrast 3: C3 <- c(0,0,1,-1,0)); and, "Is soap different than the control group?" (contrast 4: C4 <- c(0,1,0,0,-1)). Notice that all contrast coefficients add to zero.

   pnc soap cloram metro control
1   15   41     25    10      41
2   12   40     23    12      38
3   ...

The contrast matrix will be:

contrastmatrix
     C1 C2 C3 C4
[1,]  2  2  0  0
[2,] -3  0  0  1
[3,]  2 -1  1  0
[4,]  2 -1 -1  0
[5,] -3  0  0 -1

And the crossproduct confirm zeros on all off-diagonal entries:

crossprod(contrastmatrix)
   C1 C2 C3 C4
C1 30  0  0  0
C2  0  6  0  0
C3  0  0  2  0
C4  0  0  0  2

3. Why maximum number of orthogonal contrasts equals $r-1$? Geometrically, this follows from the fact that the sum square of the treatment groups $\sum_{i} n_i (\bar{y}_i - \bar{y})^2$ of the initial ANOVA can be expressed as a function of the matrix $W$ here, and the estimate of the orthogonal contrasts are proportional to scalar multiplications of the $r-1$ eigenvectors of $W$ corresponding to $\text{eigenvalues}= 1$. The last of the $r$ eigenvalues being $0$ because the trace of $Tr(W) = r-1$, and there are $r$ total eigenvalues (aside: the sum of eigenvalues equals the trace). Being that $W$ is a symmetrical and real matrix the eigenvectors associated with these eigenvectors to $\text{e-values} = 1$ will form an orthogonal set. Also they will all be orthogonal to the eigenvector corresponding to $\text{e-value}=0$. So the contrasts will correspond to linear combinations of the orthogonal basis of the $r-1$ dimensional space perpendicular to the evector to $\text{e-value} =0$, explaining the $r-1$ max rule.

Reference: Harry M. Schey. A Geometric Description of Ortogonal Contrasts in One-Way Analysis of Variance. The American Statistician, May 1985.