# Why do I not get a p-value from this ANOVA in R?

Here's the data:

> tires <- data.frame(Wear  = c(17, 14, 12, 13, 14, 14, 12, 11,
13, 13, 10, 11, 13, 8, 9, 9),
Brand = rep(LETTERS[1:4], 4),
Car   = as.character(as.roman(rep(1:4, each = 4))))
> tires
Wear Brand Car
1    17     A   I
2    14     B   I
3    12     C   I
4    13     D   I
5    14     A  II
6    14     B  II
7    12     C  II
8    11     D  II
9    13     A III
10   13     B III
11   10     C III
12   11     D III
13   13     A  IV
14    8     B  IV
15    9     C  IV
16    9     D  IV


Now I fit a two-way ANOVA with interaction:

two.way <- aov(Wear ~ Brand + Car + Brand:Car, data = tires)


Finally, no p-values:

> summary(two.way)
Df Sum Sq Mean Sq
Brand        3  30.69  10.229
Car          3  38.69  12.896
Brand:Car    9  11.56   1.285


A regular two-way ANOVA (i.e., Wear ~ Brand + Car) gives me p-values:

> summary(aov(Wear ~ Brand + Car, data = tires))
Df Sum Sq Mean Sq F value  Pr(>F)
Brand        3  30.69  10.229   7.962 0.00668 **
Car          3  38.69  12.896  10.038 0.00313 **
Residuals    9  11.56   1.285
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1


Is there a way to interpret this? The interaction plot shows me that there is definitely interaction between Brand and Car so I am hoping to incorporate this into my model.

-

## 1 Answer

Your model is saturated. Any model will use at least 1 degree of freedom. You have 2 factors with 4 levels each. They both require 3 additional degrees of freedom. The interaction consumes another 9 degrees of freedom. Summing those 1 + 3 + 3 + 9 = 16, but you have only 16 data. Thus, there are no degrees of freedom left with which to determine the residual variability, form standard errors, or test any hypotheses.

-
Thank you. Since there is a clear interaction effect here, is there any alternative analysis I can perform or am I limited to the two-way ANOVA without interaction in this case? –  John Apr 17 at 1:47
There is no way possible that it can be "clear" there is an interaction effect. I don't doubt that you interpret what you see as an interaction, but it is logically not possible to determine whether there is an interaction. You need more data. A lot more. –  gung Apr 17 at 1:49
Not sure I totally agree with you @gung. Though it's rather philosophical, I think you can have a clear interaction effect based on point estimates alone, though you will lack the ability to statistically test it. –  wannymahoots Apr 17 at 14:23
@wanny This is not a "philosophical issue." Gung is absolutely correct here: without at least one additional data value, there is no information about variability in the saturated model. The impression of an interaction can always be created in this situation simply by sorting the columns and rows appropriately: that makes such an impression merely an artifact of how one has presented the data. OTOH, if the names of brands (A,B,C,D) and cars (I,II,III,IV) had some natural or meaningful order (e.g., related to market share or price), then an interaction could be tested with these data. –  whuber Apr 17 at 15:40
Agreed, I was assuming the factors were ordered. –  wannymahoots Apr 17 at 16:11