1
$\begingroup$

I am trying to replicate a mixed ANOVA using a mixed model, but the mixed model is giving me different results. I'd like to know why, and how to get them to give the same results, assuming this is possible / advisable. If I have an incorrect premise somewhere I'd like to know about that too.

Here's my situation. I have a 3x3x3 mixed design with two within-ss factors, "section" and "format", and one between-ss factor, "condition". My dv is "accuracy", which - for a given subject, level of task, and level of format - represents percent correct out of 16 trials.

My main analysis which I'll report is a mixed ANOVA, code below.

GLM
  TRE_GRA TRE_TAB TRE_TXT INT_GRA INT_TAB INT_TXT SEC_GRA SEC_TAB SEC_TXT BY training
  /WSFACTOR = section 3 Polynomial format 3 Polynomial
  /MEASURE = accuracy
  /METHOD = SSTYPE(3)
  /CRITERIA = ALPHA(.05)
  /WSDESIGN = section format section*format
  /DESIGN = training .

For this analysis I have the data formatted for 1 row per subject, 1 column per combination of within-subjects factors. TRE, INT, and SEC are the 3 levels of section, while GRA, TAB, and TXT are the 3 levels of format. OK, this analysis works fine.

Now, I'd like to see whether an observed covariate, "influence", has any affect on accuracy after accounting for the effects in the above model, and if so, whether it mediates any of those effects. "Influence" has one value for each subject for each combination of section and format levels. I THINK that to include influence as a covariate, I need a linear mixed model rather than a regular ANCOVA, because ANCOVA would only work if there were only a single value of influence for each subject, rather than for each combination of within-ss factors. So, first, I would like to get a linear mixed model running WITHOUT influence that gives the same results as the above ANOVA, and then, once this works, see what changes when I add the covariate "influence". Sound OK so far?

OK, my problem is that the mixed model I created - without the covariate - does NOT return the same results as the above ANOVA. For this analysis there is one row per subject*section*format combination, and one column each for accuracy and "influence". Here's the code:

MIXED
  accuracy  BY training section format
  /CRITERIA = CIN(95) MXITER(100) MXSTEP(5) SCORING(1) SINGULAR(0.000000000001) HCONVERGE(0, ABSOLUTE) LCONVERGE(0, ABSOLUTE)
  PCONVERGE(0.000001, ABSOLUTE)
  /FIXED = training section format section*training format*training format*section format*section*training  | SSTYPE(3)
  /METHOD = REML
  /REPEATED = section*format | SUBJECT(subjid) COVTYPE(CS) .

The results for the between-ss factor are consistent with the original ANOVA but those for the within-ss factors are not. In particular the denominator degrees of freedom are much larger and the p values tend to be smaller, so some effects that were originally significant are no longer significant.

Advice? (1) Is my general approach OK? (2) Why isn't the mixed model giving the same results as the ANOVA? (3) How can I get it to do so, if at all? (4) If I can't, is there something else I could do that would achieve my goals?

$\endgroup$
  • $\begingroup$ Have you tried replicating a simpler analysis to see if that works? $\endgroup$ – Jeremy Miles May 30 '13 at 20:20
  • $\begingroup$ Also, do you have all of the same hypothesis tests? In the GLM SPSS will put some interactions in (I forget which), you'll need to make sure you've added them to Mixed (or removed them). $\endgroup$ – Jeremy Miles May 30 '13 at 20:20
  • $\begingroup$ (1) I have not done so with this particular project but I have done so with a very similar design in the past - the principal difference being that I now have two within-ss factors, where before I had only one. $\endgroup$ – baixiwei May 30 '13 at 20:58
  • $\begingroup$ (2) yes, the hypothesis tests are all the same. $\endgroup$ – baixiwei May 30 '13 at 20:58
  • 2
    $\begingroup$ repeated-measures Anova has assumptions (most notably sphericity) that is usually not present for mixed models. I am no expert on SPSS but I would bet this drives the difference. $\endgroup$ – Henrik Jun 5 '13 at 22:35
4
$\begingroup$

This mixed model should give you the same results as the repeated measures ANOVA

MIXED
    accuracy  BY training section format
    /CRITERIA = CIN(95) MXITER(100) MXSTEP(5) SCORING(1) SINGULAR(0.000000000001) HCONVERGE(0, ABSOLUTE) LCONVERGE(0, ABSOLUTE)  PCONVERGE(0.000001, ABSOLUTE)
    /FIXED = training section format section*training format*training format*section format*section*training  | SSTYPE(3)
    /METHOD = REML
    /Random intercept | SUBJECT(subjid)
    /Random section| SUBJECT(subjid) COVTYPE(VC)
    /Random format| SUBJECT(subjid) COVTYPE(VC) .

My guess (let me emphasize that, as I am no mixed model expert) is that the problem with your specification is that it assumes compound symmetry between all combinations of your within subject factors

$\endgroup$
  • $\begingroup$ Thanks! The results are very similar now, but still not quite identical. I notice some of the denominator dfs are larger in the LMM version. Any idea why? $\endgroup$ – baixiwei Jun 6 '13 at 19:28
  • $\begingroup$ I don't see that (I get identical DF). Is it possible that you have missing data? For repeated measures ANOVA, the entire case is dropped if you have missing data; not so for LMM (one of the main reasons that I use it). $\endgroup$ – tomr Jun 7 '13 at 20:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.