What makes a GLM estimate the means differently from the actual sample means?

Question

Disclaimer: I'll happily admit to not really knowing what I'm doing when it comes to General Mixed Models but hopefully this thread can bring me a little bit closer to some kind of understanding. If, however, this is an "OMG, you're such a n00b" question which should be closed and never spoken of again, I'll totaly understand :)

I'm running a couple of General Linear Model (using SPSS) to try to get a better feeling for the method. My dataset consists of a number of stimuli presentations with a dependent scale variable (ranging from 0 to 1) and a number of independent nominal variables for each presentation (for example the gender of the participant, a drug condition, type of stimuli et cetera).

Now, when I run a General Linear Model, I can generate a lot of pretty graphs showing both significant and non-significant main and interaction effects for all my data. When including just a couple of factors, the estimated means in these graphs are the same as if I do a descriptive analysis and just compute the means. However, when I start to add a lot of factors, the estimated means starts to deviate from the actual means in my sample.

I'm a bit confused here. What is it that makes the GLM estimate the means to be the same as the sample means and when do these estimated means start to deviate from the sample means? Does it have something to do with the number of factors, or is it about the factors themselves (for example, how they divide the data with respect to the other variables).

Answer 1

Let us have some data (shown below), predictand Y and two factors X1 and X2. X1 has 2 groups, and X2 has 3 groups. (In this particular example, the design is incomplete though, because combination X1=2 & X2=3 is absent.)

Let us run GLM command (shown). The settings are default: full factorial model, SS III type of squares, intercept present. The command requests to print out observed means for all groups of factors as well as for their combinations and also to print out the corresponding estimated means. It also saves predicted values for Y (shown below as "pre").

UNIANOVA y BY x1 x2
  /METHOD=SSTYPE(3)
  /INTERCEPT=INCLUDE
  /SAVE=PRED
  /EMMEANS=TABLES(OVERALL) 
  /EMMEANS=TABLES(x1) 
  /EMMEANS=TABLES(x2) 
  /EMMEANS=TABLES(x1*x2) 
  /PRINT=DESCRIPTIVE
  /CRITERIA=ALPHA(.05)
  /DESIGN=x1 x2 x1*x2.

     y     x1  x2      pre
  .725581   1   1   .725581
 -.147728   1   2   .046662
  .496867   1   2   .046662
 -.985803   1   2   .046662
 -.139656   1   2   .046662
 -.381405   1   2   .046662
 1.437696   1   2   .046662
  .039809   1   3  -.748909
-1.537626   1   3  -.748909
 -.402714   2   1   .159152
 1.900394   2   1   .159152
  .883087   2   1   .159152
-1.744157   2   1   .159152
 1.009084   2   2   .288968
 1.169746   2   2   .288968
  .579917   2   2   .288968
-1.022533   2   2   .288968
 -.587685   2   2   .288968
  .814123   2   2   .288968
  .003084   2   2   .288968
-1.068938   2   2   .288968
 -.175502   2   2   .288968
 1.290405   2   2   .288968
 1.166946   2   2   .288968
 -.645831   2   3  -.645831
 1.061533   3   1  1.061533
 1.143789   3   2   .676997
  .210205   3   2   .676997
 -.643339   3   3  -.360148
 -.076957   3   3  -.360148

Let us compare observed and estimated means printed out (I don't show these tables here). First, we can notice that on the lowest (the cell) level of the design, i.e. on the level of combinations of groups X1 * X2 estimated means equal observed means. This is because we had used saturated, full factorial model including all possible interactions between the factors. Second, we can see that when it comes to means on the higher, marginal, level, estimated means do not (generally) equal observed means. For example, the observed marginal mean for X1=1 is -0.05470 and the corresponding estimated mean is 0.00778.

Can we show where this difference stems from? Yes. The observed marginal mean corresponds to the simple mean of the predicted values. For X1=1, this is mean(.725581,.046662,.046662,.046662,.046662,.046662,.046662,-.748909,-.748909) = -0.05470 which is the same as the simple mean of the observed values mean(.725581,-.147728,.496867,-.985803,-.139656,-.381405,1.437696,.039809,-1.537626) = -0.05470. On the other hand, the estimated marginal mean is given by averaging the predicted values with the collapsed groups weighted equally. That is, X2=1, X2=2, X2=3 are given equal weight despite their unequal frequencies, and so 0.00778 = mean(.725581,.046662,-.748909). You may conclude yourself that if the design had been balanced - cells contained equal frequencies - estimated and observed means would have been equal to each other.

That was a simple explanation for the simple case (by "simple case" I mean defaults such as Type III SS, intercept, no covariates). You may consult "SPSS Algorithms" help document to read about how estimated, expected means are actually computed in the general case.

Answer 2

From my perspective this is completely legitimate question, which is in fact asked by many of my customers.

The mismatch can be attributed to the following:

1. Missing Data. SPSS by default excludes missing data case-wise. So if you happen to have missing observations in one factor, the cases on which the means are computed for the model (and, say, 2nd factor) are different (smaller in number) compared to those on which descriptive statistics are based.
2. Marginal means are computed with assumption, that each cell (i.e. combination of factor levels) has equal weight. This is in accordance with the more basic assumptions of ANOVA, and this is how basically ANOVA sees the data. So, if you have e.g. sex on two levels and education on two levels, the marginal mean for males is computed as $\frac{(\text{Mean of males with high education}) + (\text{Mean of males with college education})}{2}$ rather than

$\frac{(\text{Mean of males with high edu}) \cdot (\text{# males with high edu}) + (\text{Mean of males with college edu}) \cdot (\text{# males with college edu})}{\text{Number of males with any edu}}$, which equivalent to simple mean of males with any education.

This second point is equivalent to the answers given by ttnphns and Kavin Kane, but in (my opinion) easier language.

Answer 3

If you fitted a model with only one factor with the same number of levels as means you want to estimate (well, maybe one less as you have the intercept term), then the estimated means should be exactly the observed means. When you add other covariates (variables in the model) then when you estimate a least squares mean, balance is usually assumed. The means that for categorical covariates, the estimate asses that in the population, the number of individuals would be evenly spread across the levels of a covariates. In SAS, there is an option - OBSERVEDMARGINS or OM - that will give estimates based on the proportions observed in your sample, and these will usually be a lot closer to the sample means.

You have not mentioned that you are doing with the mixed or random effects. I wouldn't expect these to have a huge effect on the estimates of the means though.

I would say one thing though-if you don't understand what the methods you are using- get the help of a statistician. These things are easy to get wrong.