# Beginning anova in R: Phantom significance when testing for interaction terms

Suppose I create a dummy scenario as such:

> A <- rnorm(10000)
> B <- rnorm(10000)
> C <- rnorm(10000)
> Y <- A*B + rnorm(10000,sd=0.1)


Doing a simple ANOVA correctly identifies that none of the variables are significantly predictive of the outcome:

> anova(lm(Y~A+B+C))
Analysis of Variance Table

Response: Y
Df  Sum Sq Mean Sq F value Pr(>F)
A            1     1.5 1.54411  1.4209 0.2333
B            1     0.3 0.28909  0.2660 0.6060
C            1     1.6 1.62425  1.4946 0.2215
Residuals 9996 10862.8 1.08672


But not let's say I decide to include the interaction terms:

> anova(lm(Y~A*B*C))
Analysis of Variance Table

Response: Y
Df  Sum Sq Mean Sq    F value    Pr(>F)
A            1     1.5     1.5 1.5281e+02 < 2.2e-16 ***
B            1     0.3     0.3 2.8610e+01  9.05e-08 ***
C            1     1.6     1.6 1.6074e+02 < 2.2e-16 ***
A:B          1 10761.8 10761.8 1.0650e+06 < 2.2e-16 ***
A:C          1     0.0     0.0 9.8700e-02    0.7534
B:C          1     0.0     0.0 1.5062e+00    0.2197
A:B:C        1     0.0     0.0 1.6790e-01    0.6820
Residuals 9992   101.0     0.0
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1


It has correctly identified the interaction between A and B as being the most significant, but now for some reason the individual terms A and B have also gained significance... and C which had nothing at all to do with creating the model is significant as well? Either I have not written the test correctly or I am completely misunderstanding how a Two-Way ANOVA with interaction terms works

Using a simple linear model gives expected results:

> summary(lm(Y~A*B*C))

Call:
lm(formula = Y ~ A * B * C)

Residuals:
Min       1Q   Median       3Q      Max
-0.29566 -0.06667 -0.00092  0.06665  0.33620

Coefficients:
Estimate Std. Error  t value Pr(>|t|)
(Intercept) -0.0003212  0.0009707   -0.331    0.741
A            0.0003483  0.0009613    0.362    0.717
B            0.0003184  0.0009619    0.331    0.741
C           -0.0003213  0.0009702   -0.331    0.741
A:B          1.0008711  0.0009370 1068.214   <2e-16 ***
A:C         -0.0014855  0.0009588   -1.549    0.121
B:C          0.0008860  0.0009561    0.927    0.354
A:B:C       -0.0002489  0.0009085   -0.274    0.784
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.09705 on 9992 degrees of freedom
Multiple R-squared:  0.9913,    Adjusted R-squared:  0.9913
F-statistic: 1.634e+05 on 7 and 9992 DF,  p-value: < 2.2e-16


This is because anova() and aov() in R use Type I sums of squares by default, instead of Type III sums of squares. The workaround is clumsy, but can be achieved thus:

library(car) #need to have package "car" installed
options(contrasts=c(unordered="contr.sum", ordered="contr.poly")) #set sum-to-zero contrast

Anova(lm(Y~A+B+C), type="III") #using the Anova() function from the car package
Anova(lm(Y~A*B*C), type="III") #main effects are not significant with Type III sums of squares


For example, I used your code to generate A, B, C, and Y, and used the code above to get my ANOVA tables, and I get the following:

> Anova(lm(Y~A+B+C), type="III")
Anova Table (Type III tests)

Response: Y
Sum Sq   Df F value Pr(>F)
(Intercept)     1.3    1  1.2392 0.2657
A               0.9    1  0.9259 0.3360
B               2.0    1  2.0098 0.1563
C               0.0    1  0.0057 0.9397
Residuals   10111.8 9996


Now with the interaction terms:

> Anova(lm(Y~A*B*C), type="III")
Anova Table (Type III tests)

Response: Y
Sum Sq   Df    F value Pr(>F)
(Intercept)    0.0    1 3.7080e-01 0.5426
A              0.0    1 1.8000e-02 0.8932
B              0.0    1 2.1904e+00 0.1389
C              0.0    1 1.1700e-02 0.9139
A:B         9980.5    1 1.0013e+06 <2e-16 ***
A:C            0.0    1 2.8640e-01 0.5925
B:C            0.0    1 2.7710e-01 0.5986
A:B:C          0.0    1 1.8800e-02 0.8909
Residuals     99.6 9992
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1


There is an entry about this in the R FAQ page, along with an alternative to get to the Type III sums of squares.