Disclaimer: I'll happily admit to not really knowing what I'm doing when it comes to General Mixed Models but hopefully this thread can bring me a little bit closer to some kind of understanding. If, however, this is an "OMG, you're such a n00b" question which should be closed and never spoken of again, I'll totaly understand :)

I'm running a couple of General Linear Model (using SPSS) to try to get a better feeling for the method. My dataset consists of a number of stimuli presentations with a dependent scale variable (ranging from 0 to 1) and a number of independent nominal variables for each presentation (for example the gender of the participant, a drug condition, type of stimuli et cetera).

Now, when I run a General Linear Model, I can generate a lot of pretty graphs showing both significant and non-significant main and interaction effects for all my data. When including just a couple of factors, the estimated means in these graphs are the same as if I do a descriptive analysis and just compute the means. However, when I start to add a lot of factors, the estimated means starts to deviate from the actual means in my sample.

I'm a bit confused here. What is it that makes the GLM estimate the means to be the same as the sample means and when do these estimated means start to deviate from the sample means? Does it have something to do with the number of factors, or is it about the factors themselves (for example, how they divide the data with respect to the other variables).

  • 1
    $\begingroup$ When it comes to GLMM noone asking questions is considered pejoratively as "a noob". For what its worth, I think this applies to this site generally. $\endgroup$ – Momo Nov 3 '12 at 1:30
  • $\begingroup$ @Speldosa, General Linear model of SPSS (GLM) corresponds to "linear model" tag of this site, not "generalized liner model". So I edited your tag $\endgroup$ – ttnphns Nov 7 '12 at 11:20

Let us have some data (shown below), predictand Y and two factors X1 and X2. X1 has 2 groups, and X2 has 3 groups. (In this particular example, the design is incomplete though, because combination X1=2 & X2=3 is absent.)

Let us run GLM command (shown). The settings are default: full factorial model, SS III type of squares, intercept present. The command requests to print out observed means for all groups of factors as well as for their combinations and also to print out the corresponding estimated means. It also saves predicted values for Y (shown below as "pre").

  /DESIGN=x1 x2 x1*x2.

     y     x1  x2      pre
  .725581   1   1   .725581
 -.147728   1   2   .046662
  .496867   1   2   .046662
 -.985803   1   2   .046662
 -.139656   1   2   .046662
 -.381405   1   2   .046662
 1.437696   1   2   .046662
  .039809   1   3  -.748909
-1.537626   1   3  -.748909
 -.402714   2   1   .159152
 1.900394   2   1   .159152
  .883087   2   1   .159152
-1.744157   2   1   .159152
 1.009084   2   2   .288968
 1.169746   2   2   .288968
  .579917   2   2   .288968
-1.022533   2   2   .288968
 -.587685   2   2   .288968
  .814123   2   2   .288968
  .003084   2   2   .288968
-1.068938   2   2   .288968
 -.175502   2   2   .288968
 1.290405   2   2   .288968
 1.166946   2   2   .288968
 -.645831   2   3  -.645831
 1.061533   3   1  1.061533
 1.143789   3   2   .676997
  .210205   3   2   .676997
 -.643339   3   3  -.360148
 -.076957   3   3  -.360148

Let us compare observed and estimated means printed out (I don't show these tables here). First, we can notice that on the lowest (the cell) level of the design, i.e. on the level of combinations of groups X1 * X2 estimated means equal observed means. This is because we had used saturated, full factorial model including all possible interactions between the factors. Second, we can see that when it comes to means on the higher, marginal, level, estimated means do not (generally) equal observed means. For example, the observed marginal mean for X1=1 is -0.05470 and the corresponding estimated mean is 0.00778.

Can we show where this difference stems from? Yes. The observed marginal mean corresponds to the simple mean of the predicted values. For X1=1, this is mean(.725581,.046662,.046662,.046662,.046662,.046662,.046662,-.748909,-.748909) = -0.05470 which is the same as the simple mean of the observed values mean(.725581,-.147728,.496867,-.985803,-.139656,-.381405,1.437696,.039809,-1.537626) = -0.05470. On the other hand, the estimated marginal mean is given by averaging the predicted values with the collapsed groups weighted equally. That is, X2=1, X2=2, X2=3 are given equal weight despite their unequal frequencies, and so 0.00778 = mean(.725581,.046662,-.748909). You may conclude yourself that if the design had been balanced - cells contained equal frequencies - estimated and observed means would have been equal to each other.

That was a simple explanation for the simple case (by "simple case" I mean defaults such as Type III SS, intercept, no covariates). You may consult "SPSS Algorithms" help document to read about how estimated, expected means are actually computed in the general case.

  • $\begingroup$ Interesting! So this will mean that if I skip to include an unbalanced factor (for example, an imagined X3 factor), I'll get estimated means that are closer to the actual mean for mean(X1=1, X2=1), mean(X1=1, X2=2) and mean(X=1, X2=3)...In my design, I have a factor, F, that is highly unballanced when it comes to number of cases for each level (meaning the observed means cannot be trusted to correspond to the actual means). According to this argument, I shouldn't include such a factor if I don't want to screw up my other results, right? $\endgroup$ – Speldosa Nov 12 '12 at 0:16
  • $\begingroup$ Generally speaking, yes. ANOVA or ANCOVA "worsen" as groups become more unbalanced. This is because main effects get entangled and and they also get entangled with interactions terms. Different SS types (I, II, III) try to untangle them in different but not optimal ways. $\endgroup$ – ttnphns Nov 12 '12 at 6:36
  • 1
    $\begingroup$ Finally an answer that actually explains what these pesky estimated marginal means are (rather than promising and not delivering). I have been searching for this for so long! I will still have to figure out in what situations these estimated marginal means could be useful. $\endgroup$ – Richard Hardy Mar 6 '18 at 17:42
  • $\begingroup$ I have posted this question about the use of LS means; perhaps you will have some time to take a look at it at some point. $\endgroup$ – Richard Hardy Mar 7 '18 at 8:32

From my perspective this is completely legitimate question, which is in fact asked by many of my customers.

The mismatch can be attributed to the following:

  1. Missing Data. SPSS by default excludes missing data case-wise. So if you happen to have missing observations in one factor, the cases on which the means are computed for the model (and, say, 2nd factor) are different (smaller in number) compared to those on which descriptive statistics are based.
  2. Marginal means are computed with assumption, that each cell (i.e. combination of factor levels) has equal weight. This is in accordance with the more basic assumptions of ANOVA, and this is how basically ANOVA sees the data. So, if you have e.g. sex on two levels and education on two levels, the marginal mean for males is computed as $\frac{(\text{Mean of males with high education}) + (\text{Mean of males with college education})}{2}$ rather than

$\frac{(\text{Mean of males with high edu}) \cdot (\text{# males with high edu}) + (\text{Mean of males with college edu}) \cdot (\text{# males with college edu})}{\text{Number of males with any edu}}$, which equivalent to simple mean of males with any education.

This second point is equivalent to the answers given by ttnphns and Kavin Kane, but in (my opinion) easier language.

  • $\begingroup$ I don't think (2) is correct. One need not assume balanced design for the marginal effect estimation in a GLM to be consistent and unbiased for the population averaged marginal effect. In fact, adjustment is required to ensure this works when using stratified sampling. $\endgroup$ – AdamO Nov 8 '12 at 18:28
  • $\begingroup$ True. What I have written is how the SPSS (now: PASW) software is working by default in a simple language. After all, there is this disclaimer before the question. $\endgroup$ – Adam Ryczkowski Nov 8 '12 at 22:06
  • $\begingroup$ @AdamRyczkowski Actually, they have changed the name back again. They now call it "IBM SPSS". $\endgroup$ – Speldosa Nov 12 '12 at 0:03

If you fitted a model with only one factor with the same number of levels as means you want to estimate (well, maybe one less as you have the intercept term), then the estimated means should be exactly the observed means. When you add other covariates (variables in the model) then when you estimate a least squares mean, balance is usually assumed. The means that for categorical covariates, the estimate asses that in the population, the number of individuals would be evenly spread across the levels of a covariates. In SAS, there is an option - OBSERVEDMARGINS or OM - that will give estimates based on the proportions observed in your sample, and these will usually be a lot closer to the sample means.

You have not mentioned that you are doing with the mixed or random effects. I wouldn't expect these to have a huge effect on the estimates of the means though.

I would say one thing though-if you don't understand what the methods you are using- get the help of a statistician. These things are easy to get wrong.

  • $\begingroup$ Thanks for your answer! However, I'm having some trouble understanding some parts of your explanation. The problem is that I get an interaction effect for interaction xyz when just including these factors, but that this interaction dissapears when I include the factors a, b, x, y, z. You are talking about certain options in SAS to give other estimates, but what happens to the significance testing in that case? I assume I just can't walk in, change how the estimated means are computed, and force the analysis to get significant. Also, I'm just using fixed factors (that is, no random factors). $\endgroup$ – Speldosa Nov 3 '12 at 23:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.