2
$\begingroup$

enter image description here

Here we have design matrix for one-way ANOVA(effect coding), where first column consists of grand mean, and other other columns show the influence of group 1-3.

What i cannot understand: why do we have in last row three $-1$? How does it help us to interpret result? Can you provide an example of interpretation of ANOVA with effect coding?

$\endgroup$
0
$\begingroup$

why do we have in last row three −1?

The coding of the design matrix is a result of sum to 0 constraint, i.e. $\sum_{i=1}^4 \alpha_i=0$. The reason the last row is like that is because $\alpha_4=-\alpha_1-\alpha_2-\alpha_3$. Therefore, we formulate the effect form $y_4=\mu+\alpha_4=\mu-\alpha_1-\alpha_2-\alpha_3$

How does it help us to interpret result?

A re-formulation of the question is to ask "why we use sum to 0 constraint?". The sum to 0 constraint helps identifiability. If for example we encoded the design matrix as below

$$ \begin{bmatrix} y_{1,1} \\ y_{1,2} \\ y_{2,1} \\ y_{2,2} \\ y_{3,1} \\ y_{3,2} \\ y_{4,1} \\ y_{4,2} \end{bmatrix} = \begin{bmatrix} 1&1 & 0 & 0 & 0 \\ 1&1 & 0 & 0 & 0 \\ 1&0 & 1 & 0 & 0 \\ 1&0 & 1 & 0 & 0 \\ 1&0 & 0 & 1 & 0 \\ 1&0 & 0 & 1 & 0 \\ 1&0 & 0 & 0 & 1\\ 1&0 & 0 & 0 & 1 \end{bmatrix} \times \begin{bmatrix} \mu\\ \alpha_1 \\ \alpha_2 \\ \alpha_3 \\ \alpha_4 \\ \end{bmatrix} + \begin{bmatrix} \epsilon_{1,1} \\ \epsilon_{1,2} \\ \epsilon_{2,1} \\ \epsilon_{2,2} \\ \epsilon_{3,1} \\ \epsilon_{3,2} \\ \epsilon_{4,1} \\ \epsilon_{4,2} \end{bmatrix} $$

Note that the problem is over-parameterised, ie there are 4 groups but 5 parameters. The additional parameter makes it possible for the solution to be non-unique, therefore, we must include the additional contstraint

Can you provide an example of interpretation of ANOVA with effect coding?

Below are artificially produced the data to be able to be predicted our model exactly. As you can see, there are two solutions that I came up with. Note: in the first solution, I $\sum_{i=1}^4 \alpha_i=0$ and $\mu$ is the grand mean of all four groups.

$$ \begin{bmatrix} 2\\ 2 \\ 4 \\ 4\\ 6\\ 6 \\ 4\\ 4 \end{bmatrix} = \begin{bmatrix} 1&1 & 0 & 0 & 0 \\ 1&1 & 0 & 0 & 0 \\ 1&0 & 1 & 0 & 0 \\ 1&0 & 1 & 0 & 0 \\ 1&0 & 0 & 1 & 0 \\ 1&0 & 0 & 1 & 0 \\ 1&0 & 0 & 0 & 1\\ 1&0 & 0 & 0 & 1 \end{bmatrix} \times \begin{bmatrix} 4\\ -2 \\ 0 \\ 2 \\ 0\\ \end{bmatrix} = \begin{bmatrix} 1&1 & 0 & 0 & 0 \\ 1&1 & 0 & 0 & 0 \\ 1&0 & 1 & 0 & 0 \\ 1&0 & 1 & 0 & 0 \\ 1&0 & 0 & 1 & 0 \\ 1&0 & 0 & 1 & 0 \\ 1&0 & 0 & 0 & 1\\ 1&0 & 0 & 0 & 1 \end{bmatrix} \times \begin{bmatrix} 2\\ 0 \\ 2 \\ 4 \\ 2\\ \end{bmatrix} $$

When we impose the additional constraint, we simply are stuck with the first solution which is the only solution. But, the design matrix can be written in an alternative form as shown in the second equality below.

$$ \begin{bmatrix} 2\\ 2 \\ 4 \\ 4\\ 6\\ 6 \\ 4\\ 4 \end{bmatrix} = \begin{bmatrix} 1&1 & 0 & 0 & 0 \\ 1&1 & 0 & 0 & 0 \\ 1&0 & 1 & 0 & 0 \\ 1&0 & 1 & 0 & 0 \\ 1&0 & 0 & 1 & 0 \\ 1&0 & 0 & 1 & 0 \\ 1&0 & 0 & 0 & 1\\ 1&0 & 0 & 0 & 1 \end{bmatrix} \times \begin{bmatrix} 4\\ -2 \\ 0 \\ 2 \\ 0\\ \end{bmatrix} = \begin{bmatrix} 1&1 & 0 & 0 \\ 1&1 & 0 & 0 \\ 1&0 & 1 & 0 \\ 1&0 & 1 & 0 \\ 1&0 & 0 & 1 \\ 1&0 & 0 & 1 \\ 1&-1 & -1 & -1 \\ 1&-1 & -1 & -1 \end{bmatrix} \times \begin{bmatrix} 4\\ -2 \\ 0 \\ 2 \end{bmatrix} $$

By imposing the additional constraint, the number of columns of my design matrix reduces by 1 since $\alpha_4$ is determined by the constraint I set. The interpretation of $\alpha_i$ for $1 \leq i \leq 3 $, is that the coefficients are the additive effect on top of the grand mean. Can we find another solution for $\mu, \alpha_1, \alpha_2, \alpha_3$ using the sum constraint? Well we can reduce the form as below.

$$ \begin{bmatrix} 2\\ 4 \\ 6\\ 4 \end{bmatrix} = \begin{bmatrix} 1&1 & 0 & 0 \\ 1&0 & 1 & 0 \\ 1&0 & 0 & 1 \\ 1&-1 & -1 & -1 \end{bmatrix} \times \begin{bmatrix} 4\\ -2 \\ 0 \\ 2 \end{bmatrix} $$

As you can see, this is a square matrix of full rank therefore, it is invertible and the solution is unique.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.