5
$\begingroup$

I have a general, presumably simple, question, but I couldn't find a conclusive answer so far.

Assume I have a simple case of a General Linear Model with one categorical predictor variable, that has 3 levels. This corresponds to a one-way ANOVA with 3 groups.

The Linear Model would then be: $Y_i = \beta_0 + \beta_1X_1 + \beta_2X_2$

I use the following standard Helmert contrast codes (I, II) for the three groups A, B, C: 

     A    B    C
I   -2    1    1
II   0   -1    1

Based on these contrast codes, will the intercept of the linear model represent the mean of all group means? More generally, does this always apply to Helmert contrast coded variables, regardless of the number of predictor variables or variable levels?

Obviously, the intercept represents the predicted value, when all predictor variables (in this case $X_1$ and $X_2$) equal zero, but I can't figure out if that represents the mean of the group means.

Improve this question
$\endgroup$
5
  • $\begingroup$ Does that help: ats.ucla.edu/stat/r/library/contrast_coding.htm? $\endgroup$
    – chl
    Nov 24, 2013 at 13:08
  • $\begingroup$ I've come across this site before, but unfortunately it doesn't say whether the intercept in Helmert coding equals the mean of all group means. $\endgroup$
    – vincentqu
    Nov 24, 2013 at 14:26
  • 5
    $\begingroup$ Yes it is the case. If you write down regression equations based on the contrast matrix returned by R and try to identify each regression coefficient, you will find that $\beta_0$ is the mean of all levels (like in deviation or sum contrast) while $\beta_i,\, i=1\dots j$, represent sequential comparisons between adjacent categories (2 vs. 1, 3 vs. {1,2}, etc.). $\endgroup$
    – chl
    Nov 24, 2013 at 15:19
  • $\begingroup$ Thanks! Does that also hold, when I have more than 1 categorical variable and consequently my model contains interaction terms? $\endgroup$
    – vincentqu
    Nov 24, 2013 at 15:31
  • 2
    $\begingroup$ @chl, why not make that an official answer & show how it's done? $\endgroup$
    – gung - Reinstate Monica
    Nov 24, 2013 at 20:04

1 Answer 1

Reset to default
2
$\begingroup$

Yes. You can easily verify this by carrying out the following steps:

First, express the means of $A$, $B$, and $C$, in terms of the model with the specified contrast: \begin{eqnarray*} 1\hat{\beta}_{0}-2\hat{\beta}_{1}+0\hat{\beta}_{2} & = & \hat{\mu}_{A}=E(Y_{A})\\ 1\hat{\beta}_{0}+1\hat{\beta}_{1}-1\hat{\beta}_{2} & = & \hat{\mu}_{B}=E(Y_{B})\\ 1\hat{\beta}_{0}+1\hat{\beta}_{1}+1\hat{\beta}_{2} & = & \hat{\mu}_{C}=E(Y_{C}) \end{eqnarray*}

Here, each $\hat{\mu}_i$ represents the group mean of group $i$, $i=A, B, C$. Next, place each beta coefficient into a matrix augmented with the means on the right, and place the matrix in row-reduced echelon form using Guass-Jordan elimination:

\begin{eqnarray*} \begin{bmatrix}1 & -2 & 0 & | & \hat{\mu}_{A}\\ 1 & 1 & -1 & | & \hat{\mu}_{B}\\ 1 & 1 & 1 & | & \hat{\mu}_{C} \end{bmatrix} & \sim & \begin{bmatrix}1 & -2 & 0 & | & \hat{\mu}_{A}\\ 0 & 3 & -1 & | & \hat{\mu}_{B}-\hat{\mu}_{A}\\ 0 & 3 & 1 & | & \hat{\mu}_{C}-\hat{\mu}_{A} \end{bmatrix}\\ & \sim & \begin{bmatrix}1 & -2 & 0 & | & \hat{\mu}_{A}\\ 0 & 3 & -1 & | & \hat{\mu}_{B}-\hat{\mu}_{A}\\ 0 & 0 & 2 & | & \left(\hat{\mu}_{C}-\hat{\mu}_{A}\right)-\left(\hat{\mu}_{B}-\hat{\mu}_{A}\right) \end{bmatrix}\\ & \sim & \begin{bmatrix}1 & -2 & 0 & | & \hat{\mu}_{A}\\ 0 & 3 & -1 & | & \hat{\mu}_{B}-\hat{\mu}_{A}\\ 0 & 0 & 1 & | & \frac{1}{2}\left[\left(\hat{\mu}_{C}-\hat{\mu}_{A}\right)-\left(\hat{\mu}_{B}-\hat{\mu}_{A}\right)\right] \end{bmatrix}\\ & \sim & \begin{bmatrix}1 & -2 & 0 & | & \hat{\mu}_{A}\\ 0 & 1 & 0 & | & \frac{1}{3}\left\{ \left(\hat{\mu}_{B}-\hat{\mu}_{A}\right)+\frac{1}{2}\left[\left(\hat{\mu}_{C}-\hat{\mu}_{A}\right)-\left(\hat{\mu}_{B}-\hat{\mu}_{A}\right)\right]\right\} \\ 0 & 0 & 1 & | & \frac{1}{2}\left[\left(\hat{\mu}_{C}-\hat{\mu}_{A}\right)-\left(\hat{\mu}_{B}-\hat{\mu}_{A}\right)\right] \end{bmatrix}\\ & \sim & \begin{bmatrix}1 & 0 & 0 & | & \hat{\mu}_{A}+\frac{2}{3}\left\{ \left(\hat{\mu}_{B}-\hat{\mu}_{A}\right)+\frac{1}{2}\left[\left(\hat{\mu}_{C}-\hat{\mu}_{A}\right)-\left(\hat{\mu}_{B}-\hat{\mu}_{A}\right)\right]\right\} \\ 0 & 1 & 0 & | & \frac{1}{3}\left\{ \left(\hat{\mu}_{B}-\hat{\mu}_{A}\right)+\frac{1}{2}\left[\left(\hat{\mu}_{C}-\hat{\mu}_{A}\right)-\left(\hat{\mu}_{B}-\hat{\mu}_{A}\right)\right]\right\} \\ 0 & 0 & 1 & | & \frac{1}{2}\left[\left(\hat{\mu}_{C}-\hat{\mu}_{A}\right)-\left(\hat{\mu}_{B}-\hat{\mu}_{A}\right)\right] \end{bmatrix} \end{eqnarray*}

So, now, we know that the first pivot position corresponds to:

\begin{eqnarray*} \hat{\beta}{}_{0} & = & \hat{\mu}_{A}+\frac{2}{3}\left\{ \left(\hat{\mu}_{B}-\hat{\mu}_{A}\right)+\frac{1}{2}\left[\left(\hat{\mu}_{C}-\hat{\mu}_{A}\right)-\left(\hat{\mu}_{B}-\hat{\mu}_{A}\right)\right]\right\} \\ & = & \hat{\mu}_{A}-\frac{2}{3}\hat{\mu}_{A}-\frac{1}{3}\hat{\mu}_{A}+\frac{1}{3}\hat{\mu}_{A}+\frac{2}{3}\hat{\mu}_{B}-\frac{1}{3}\hat{\mu}_{B}+\frac{1}{3}\hat{\mu}_{C}\\ & = & \frac{1}{3}\hat{\mu}_{A}+\frac{1}{3}\hat{\mu}_{B}+\frac{1}{3}\hat{\mu}_{C}\\ & = & \frac{\hat{\mu}_{A}+\hat{\mu}_{B}+\hat{\mu}_{C}}{3} \end{eqnarray*}

The final expression indicates that $\hat{\beta}{}_{0}$, the intercept, represents the simple mean of the group means.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.