# ANOVA results discrepancy between R and SPSS

I ran into problems when trying to double check SPSS ANOVA calculations with R. I am aware of the difference in defaults concerning the Sum of Squares, therefore I followed guidelines for aligning SPSS and R results. After failing to get equivalent results I used a toy data set to make sure it is not a problem specific to the data involved.

Here is the toy data:

Crit    A   B
1   1   1
2   1   1
3   1   0
4   1   0
4   1   0
5   1   0
6   1   0
6   1   0
7   0   1
8   0   1
9   0   1
10  0   1
11  0   1
12  0   0
13  0   0
12  0   0


When running the ANOVA using SPSS (24.0.0.0, 64-bit) I use the Syntax:

UNIANOVA Crit BY A B
/METHOD=SSTYPE(3)
/INTERCEPT=INCLUDE
/CRITERIA=ALPHA(.05)
/DESIGN=A B A*B.


This results in the (shortened) output:

Tests of Between-Subjects Effects

Dependent Variable:   Crit

Source  SS      df     MS     F      Sig.
Itc.  630.208    1 630.208  408.784 .000
A     191.690    1 191.690  124.339 .000
B      35.208    1  35.208   22.838 .000
A * B    .023    1    .023     .015 .905
Error  18.500   12   1.542


In R (RStudio V. 1.1.383, R 3.4.3), I used the "car" package, using the code:

simpledata<-read.table("simpleData.txt",sep="\t", header=T)
options(contrasts = c("contr.sum", "contr.poly"))
model=lm(Crit ~ B+A+B:A, data = simpledata)
Anova(model, type="3")


This should produce equivalent numbers based on multiple guides found online. Instead, I find the output:

Response: Crit
Sum Sq Df F value    Pr(>F)
(Intercept) 456.33  1 296.000 8.026e-10 ***
A           117.56  1  76.252 1.517e-06 ***
B            20.83  1  13.514  0.003172 **
A:B           0.02  1   0.015  0.904503
Residuals    18.50 12


As can be seen, the results for the interaction and the error are identical, but nothing else aligns. This matches the comparison for my original problem. Notice that I receive the following output when changing the SS to "2":

Response: Crit
Sum Sq Df F value    Pr(>F)
A         197.890  1 128.361 9.151e-08 ***
B          35.852  1  23.255 0.0004174 ***
A:B         0.023  1   0.015 0.9045030
Residuals  18.500 12


This is closer but not identical to the SPSS output (but it should not be, as SPSS uses Type III).

Has anyone else run into this problem and found a satisfactory solution? For psychologists this should be a deeply concerning question....

It's all good. You just need to tell R that you want A and B to be factors (if you want to do an Analysis of Variance). Currently, R treats A and B as numeric variables (or, more precisely, as integers), which will run a multiple linear regression analysis.

class(simpledata$$A) # [1] "integer" class(simpledata$$B)
# [1] "integer"

# CONVERT A AND B TO FACTORS
simpledata$$Af <- as.factor(simpledata$$A)
simpledata$$Bf <- as.factor(simpledata$$B)

# FIT MODEL WITH FACTOR VARIABLES
model=lm(Crit ~ Bf + Af + Bf:Af, data = simpledata)
Anova(model, type = "3")

#Anova Table (Type III tests)
#Response: Crit
#            Sum Sq Df F value    Pr(>F)
#(Intercept) 630.21  1 408.784 1.229e-10 ***
#Bf           35.21  1  22.838 0.0004494 ***
#Af          191.69  1 124.339 1.091e-07 ***
#Bf:Af         0.02  1   0.015 0.9045030
#Residuals    18.50 12

• This solved both problems (the toy and the real-world one). I still wonder how many results out there are based on surprising variants... – jank Nov 6 '18 at 22:14
• @jank That's a good question and it looks like others have indeed ran into this before. This highlights again that cross-checking your results can save you life ;) - especially when switching from one statistical software package to another. – Stefan Nov 7 '18 at 1:53