4
$\begingroup$

I've spent days trying to determine the difference between these two methods, which, as best I can tell, represent the univariate and multivariate approaches to a within-subjects design. Can anybody tell me why these two analyses get different F-values for their effects?

Here's demonstration data in the shape that repeated-measures uses it:

Data Wide;
input subject   DV11    DV12    DV13    DV21    DV22    DV23    DV31    DV32    DV33 ;
cards;
1   36  52  55  60  68  64  40  42  44
2   30  32  34  40  42  44  20  22  24
3   -10 -8  -6  0   8   6   -30 -25 -15
;
run;

And the analysis:

PROC GLM data=wide;
class subject;
model dv11 dv12 dv13 dv21 dv22 dv23 dv31 dv32 dv33 = /nouni;
REPEATED factor1 3, factor2 3;
RUN;

Factor 1 has F(2,4) = 48.35; Factor 2 has F(2,4) = 11.5; interaction has F(4,8) = 0.65.

For the other approach, which I'm assuming is the multivariate approach (correct me if I'm wrong!), we read the data in as a flat vector labeled with the levels of the within-subject factors:

Data Narrow;
input subject   factor1 factor2 value;
cards;
1   1   1   36
1   1   2   52
1   1   3   55
1   2   1   60
1   2   2   68
1   2   3   64
1   3   1   40
1   3   2   42
1   3   3   44
2   1   1   30
2   1   2   32
2   1   3   34
2   2   1   40
2   2   2   42
2   2   3   44
2   3   1   20
2   3   2   22
2   3   3   24
3   1   1   -10
3   1   2   -8
3   1   3   -6
3   2   1   0
3   2   2   8
3   2   3   6
3   3   1   -30
3   3   2   -25
3   3   3   -15
;
run;

and the analysis:

    PROC GLM data=Narrow;
class subject factor1 factor2;
model value = factor1|factor2 subject;
run;

This time our results are:
Factor 1 F(2,16) = 77.57, Factor 2 F(2,16) = 7.70, interaction F(4,16) = 0.56.

Can anybody explain why these analyses get different results? I am more comfortable with the second method, but the first is traditionally preferred in my field.

UPDATE: As best I can tell, the differences are that:

  1. PROC GLM is based on Ordinary Least Squares, while PROC MIXED uses maximum likelihood. This difference is why MIXED can handle missing obs but REPEATED cannot.

  2. A repeated-measures analysis analyzes the pairwise differences, while mixed-effects does not. Repeated-measures thereby involves an assumption of sphericity (constant variance of pairwise differences) while MIXED assumes normality of the raw observations (?)

  3. Calculation of degrees of freedom are different between models, perhaps because GLM REPEATED is more conservative than MIXED due to concerns re: violations of sphericity (?)

Sources: http://www.ats.ucla.edu/stat/sas/library/GLMvsMIXED_os.htm and http://www.ats.ucla.edu/stat/sas/library/mixedglm.pdf

$\endgroup$
1
$\begingroup$

I can't tell you what the differences are so I'm not directly answering your question. I don't know what your first model is doing and for the second model subject should be a random effect. Anyways you really should not use proc GLM for mixed models, instead use proc mixed or proc glimmix.

Here is how I would analyze it, which accounts for random intercepts for each subject:

proc glimmix;
class factor1 factor2 subject;
model value=factor1|factor2/ddfm=kr solution;
lsmeans factor1|factor2/ lines adjust=tukey;
random subject;
run;

And the output (which actually matches up with your second set of results!):

enter image description here

enter image description here

enter image description here

$\endgroup$
  • $\begingroup$ I think it's interesting that whether a fixed or random effects of subject is fit, it seems to extract the same amount of error variance. Don't you? Naturally, I agree that a random effect of subject is the correct way to run model 2. I'm still curious about model 1, though, given that most of the (older) members of my department use repeated-measures ANOVA for everything... $\endgroup$ – CoolBuffScienceDude Apr 1 '14 at 18:38
  • $\begingroup$ Yes it is interesting in this case the two approaches are practically identical but this is probably more of just a coincidence due to the perfectly balanced data and possibly other factors. $\endgroup$ – Glen Apr 1 '14 at 18:53
  • $\begingroup$ I don't believe we should consider Subject as a factor. After all, if for each combination of factor, you only have one measurement, what is the use of statistics? $\endgroup$ – Dirk Horsten Aug 5 '15 at 11:31
  • $\begingroup$ The estimates are the same whether or not the subject is included but inference will be wrong without the subject effect. Degrees of freedom and standard errors would not be correct. $\endgroup$ – Glen Aug 5 '15 at 16:48
1
$\begingroup$

I came across your question while reading the chapter Analysis of variance: repeated measures design in Making Sense of Statistics in Psychology from Brian S Everitt.

I tried to reproduce Table 5.13: Analysis of variance for reverse Stroop Experiment. The experiment wants to discover if a field (in-)dependent cognitive style (Cognition) influences performance (ReadTime) for the Stroop effect test for Words about Forms and Colors in Neutral, Congruent and Incongruent conditions.

Here is what I got

title2 'Your colleagues preference: Repeated measurement';
proc glm data=psyData.Stroop_Rep;
    Class Cognition;
    model Color_: Form_: = Cognition /nouni;
    repeated Word 2, Condition;
    format Word WordType. Condition Condition.;
run;

Repeated Measurement

The (1), (2) are referred to later. The values in blue are the results in the book. The results are comparable, but I guess there is some detail I did not understand. The factor of interest, Cognition is not really significant. In a report one would write more research is needed.

title2 'Your preference: Subject as a factor';
proc glm data=psyData.Stroop;
    class Cognition Word Condition;
    model ReadTime = Word|Condition|Cognition Subject ;
run;

As the Repeated measurement approach gave Type III results, let us look at these first:

Type III

I illustrated both report on the same things by adding cross references (1) etc., but the results are quite different:

  • Word type became much more significant and started interacting with Cognition
  • Condition lost some significance
  • Cognition itself, that what actually matters, fell completely out
  • and of course Subject appeared en the equation,

Now let us also look at the Type I results:

Type I

Now we see Cognition becomes extremely significant for you.

So you really come to different conclusions than your colleagues. A quick look at the data shows that Cognition

If you tell SAS Subject is a factor, knowing that Cognition is just an attribute of the subjects, you actually introduce an important collinearity in your model, which makes the result very unstable.

Now let us do what we should have done at the first place: look at the data

enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.