3
$\begingroup$

I've just compared the ANOVA tables generated by SPSS and Statistica with the aov table provided by summary(aov.model). They yield identical between Subject effects (e.g., NativeLanguage(English vs Other), but different F ratios for within subject effects (e.g., Class(animate vs inanimate), aov F-ratios being consistently smaller and more conservative. The interaction of the within-between factors again yields identical terms. I am stumped. HOw can this be? Any suggestion? Here are more details:

I was analyzing the RTs lexdec data base from the languageR package. I did some minor data adjustments. E.g., To get a sense for RTs I reversed the log transform exp(lexdec$RT) then removed Error responses and RT outliers. Using ddply I obtained condition means for NativeLanguage and Class for each subject (the data frame is shown at the bottom ob my post). Analyzing these data, I obtained different aov and STATISTICA summaries. Specifically for the within-subjects factor Class was p~.18 with aov and p~.06 with STATISTICA and SPSS, with larger Class SS values shown by the two commercial packages (569) than by aov (301).

I've tried to make the two anova outputs (shown below) look transparent but it appears that the posting format does not match the format shown in the question window.

> C1.anova <- aov(RT ~ (Class * NativeLanguage)
+  + Error(Subject/Class) + (NativeLanguage), data=C1 )
> summary(C1.anova)

Error: Subject
               Df Sum Sq Mean Sq F value  Pr(>F)  
NativeLanguage  1  81413   81413  6.2973 0.02131 *
Residuals      19 245637   12928                  
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 

Error: Subject:Class
                     Df  Sum Sq Mean Sq F value   Pr(>F)   
Class                 1  301.51  301.51  1.9661 0.176994   
Class:NativeLanguage  1 2175.86 2175.86 14.1880 0.001305 **
Residuals            19 2913.83  153.36                    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 

STATISTICA

                       SS  DF        MS         F          p
Intercept        15387240   1  15387240  1190.202   0.000000
NativeLanguage      81413   1     81413     6.297   0.021311
Error              245637  19     12928     
Class                 569   1       569     3.709   0.069217
Class*NativeLanguage 2176   1      2176    14.188   0.001305
Error                2914  19       153     

Data:

   Subject  Class NativeLanguage       RT
1       A1 animal        English 557.6410
2       A1  plant        English 548.4687
3       A2 animal        English 533.4737
4       A2  plant        English 511.7941
5       A3 animal          Other 598.9545
6       A3  plant          Other 602.4118
7        C animal        English 562.8864
8        C  plant        English 560.0588
9        D animal          Other 630.1464
10       D  plant          Other 604.0286
11       I animal          Other 542.1219
12       I  plant          Other 533.1666
13       J animal          Other 565.4324
14       J  plant          Other 513.2333
15       K animal        English 492.4500
16       K  plant        English 517.7333
17      M1 animal        English 481.8372
18      M1  plant        English 497.9687
19      M2 animal          Other 671.6666
20      M2  plant          Other 655.8750
21       P animal          Other 640.7209
22       P  plant          Other 610.0286
23      R1 animal        English 552.9744
24      R1  plant        English 545.4242
25      R2 animal        English 636.8864
26      R2  plant        English 675.1714
27      R3 animal        English 607.8572
28      R3  plant        English 614.9428
29       S animal        English 599.9285
30       S  plant        English 586.6286
31      T1 animal        English 580.0500
32      T1  plant        English 583.2857
33      T2 animal          Other 892.5526
34      T2  plant          Other 862.1000
35       V animal          Other 736.2619
36       V  plant          Other 718.3529
37      W1 animal        English 517.0465
38      W1  plant        English 539.2727
39      W2 animal        English 639.1363
40      W2  plant        English 666.7143
41       Z animal          Other 725.3750
42       Z  plant          Other 706.2069
$\endgroup$
  • 1
    $\begingroup$ FYI, the formula supplied to aov() has redundant terms; if you have Class*NativeLanguage, you don't need +NativeLanguage $\endgroup$ – Mike Lawrence Oct 6 '11 at 16:12
  • 1
    $\begingroup$ This question also addresses the issue of Type III SS in R: stats.stackexchange.com/q/4544/3601 $\endgroup$ – Aaron Oct 7 '11 at 13:56
2
$\begingroup$

This may be because your between-groups variable, NativeLanguage, is unbalanced (12 English, 9 Other), in which case the type of Sums-of-Squares employed is going to affect the F values. By default, aov() uses Type 1 sums of squares, which isn't recommended with unbalanced designs. Instead, use the ezANOVA() function from the ez package:

my_anova = ezANOVA(
    data = C1
    , dv = .(RT)
    , wid = .(Subject)
    , within = .(Class)
    , between = .(NativeLanguage)
    , type = 3 #SPSS uses type 3 Sums-of-Squares
    , observed = .(NativeLanguage) #ensures appropriate effect size is computed
)
#note warning about data imbalance
print(my_anova)

This yields the results table:

$ANOVA
                Effect DFn DFd         F           p p<.05          ges
2       NativeLanguage   1  19  6.297322 0.021311506     * 0.2451177279
3                Class   1  19  1.976662 0.175885891       0.0009118545
4 NativeLanguage:Class   1  19 14.187926 0.001305329     * 0.0065510116

Which strangely still has a different report for the Class effect than what you're getting from SPSS/Statistica. Adding a detailed=TRUE argument to the ezANOVA() call above gives us a slightly more detailed results table (including the intercept and sums of squares):

$ANOVA
                Effect DFn DFd          SSn        SSd          F            p p<.05          ges
1          (Intercept)   1  19 7717585.5514 245636.967 596.954632 8.136647e-16     * 0.9587389575
2       NativeLanguage   1  19   81413.4195 245636.967   6.297322 2.131151e-02     * 0.2451177279
3                Class   1  19     303.1399   2913.831   1.976662 1.758859e-01       0.0009118545
4 NativeLanguage:Class   1  19    2175.8535   2913.831  14.187926 1.305329e-03     * 0.0065510116

This shows that the mismatch lies in the SSn for the class effect; ezANOVA (which uses car::Anova()) obtains an SSn of 303ish whereas SPSS/Statistica obtain an SSn of 569.

$\endgroup$
  • $\begingroup$ Ok, I had to leave for lunch and thought the type 3 sum of squares was THE answer. However, I just had a chance to do the ezANOVA, as suggested. It ran, as specified, but I still get the same F and p effects as I do get with the aov function and the within subjects effect differs from the F and p that SPSS gives me. $\endgroup$ – alwin Oct 6 '11 at 19:05
  • $\begingroup$ Yes, I updated my answer subsequent to its first posting to note that there's still a disparity that I can't track down without access to SPSS. $\endgroup$ – Mike Lawrence Oct 6 '11 at 20:51
2
$\begingroup$

As Mike already pointed out, the difference is due to the choice for the sum of squares (III for SPSS, I for R). The important thing to add is that type III SS require a specific coding scheme for categorical variables: it needs to be of the sum-to-zero kind, e.g., effect coding or Helmert coding. R's default is dummy coding, where the coefficients do not sum to zero (compare contr.sum(4), contr.helmert(4) with contr.treatment(4)). This will cause wrong results for type III SS.

I'll use car's Anova() function for type III SS. It requires data in wide format.

> Cw <- reshape(C1, direction="wide", v.names="RT", timevar="Class",
+               idvar=c("Subject", "NativeLanguage"))

> options(contrasts=c("contr.sum", "contr.poly"))              # switch to effect coding
> library(car)                                                 # for Anova()
> fitSPFpq <- lm(cbind(RT.animal, RT.plant) ~ NativeLanguage, data=Cw)
> inSPFpq  <- data.frame(Class=gl(2, 1))
> summary(Anova(fitSPFpq, idata=inSPFpq, idesign=~Class, type="III"), multivariate=FALSE)
Univariate Type III Repeated-Measures ANOVA Assuming Sphericity

                           SS num Df Error SS den Df         F    Pr(>F)    
(Intercept)          15387240      1   245637     19 1190.2018 < 2.2e-16 ***
NativeLanguage          81413      1   245637     19    6.2973  0.021312 *  
Class                     569      1     2914     19    3.7090  0.069216 .  
NativeLanguage:Class     2176      1     2914     19   14.1879  0.001305 **

The results match those from SPSS and Statistica also for the Class effect.

(As a sidenote: some regard the dependence of type III SS in unbalanced Anova designs on the - arbitrary - choice for a coding scheme as a significant downside. The dependence comes from the chosen model comparisons which do not follow the principle of marginality.)

$\endgroup$
  • 1
    $\begingroup$ @alwin: caracal's answer explains the difference well, but it should be noted that this is more than a technical problem of getting the softare to agree; you should think about your scientific questions and decide which is more appropriate. In most cases that I've done, the Type I or II (which are the same here, I think) is more appropriate than Type III, but you should think about your questions and decide for yourself. $\endgroup$ – Aaron Oct 7 '11 at 13:52
  • $\begingroup$ @Aaron Good points about the equivalence of type I and type II tests in this case, and about the different SS types effectively testing different hypotheses! $\endgroup$ – caracal Oct 7 '11 at 16:24
  • $\begingroup$ Thank you very much, Caracal and Aaron. This additional information is extremely helpful. My sense is that Type 1 is more appropriate in this case. In fact, when I computed NativeLanguage and Class condition means for each word of the data set and then used 'Words' as random factor (often done in psycholinguistic studies), then the F value for Class was less than 1 -- meaning Class has no real effect. $\endgroup$ – alwin Oct 7 '11 at 17:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.