3
$\begingroup$

As I understand it aov() is a Type I SS ANOVA. I have followed online advice on how to change its output to Type III.

I have also run a Type III ezANOVA.

My problem is that the F-values and corresponding p-values differ greatly between these two approaches.

Here is reproducible code for my data set:

Cond_Per_Row_stats<-structure(list(Coherence = structure(c(1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L,
3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L,
3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L,
3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L,
3L, 3L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L,
2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L,
2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L,
2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 5L, 5L,
5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L,
5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L,
5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L,
5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 4L, 4L, 4L, 4L, 4L, 4L,
4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L,
4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L,
4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L,
4L, 4L, 4L, 4L, 4L, 4L), .Label = c("P0.0", "P3", "P35", "P4",
"P45"), class = "factor"), PrimeDuration = structure(c(1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L,
2L, 2L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 2L, 2L, 2L, 2L,
2L, 2L, 2L, 2L, 2L, 2L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L,
2L, 2L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 2L, 2L, 2L, 2L,
2L, 2L, 2L, 2L, 2L, 2L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L,
2L, 2L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 2L, 2L, 2L, 2L,
2L, 2L, 2L, 2L, 2L, 2L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L,
2L, 2L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 2L, 2L, 2L, 2L,
2L, 2L, 2L, 2L, 2L, 2L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L), .Label = c("1200ms",
"50ms"), class = "factor"), PrimeType = structure(c(1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L,
2L, 2L, 2L, 2L, 2L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L,
3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 2L, 2L, 2L,
2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L,
2L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L,
3L, 3L, 3L, 3L, 3L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 2L, 2L, 2L, 2L, 2L, 2L, 2L,
2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 3L, 3L, 3L,
3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L,
3L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L,
2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 3L, 3L, 3L, 3L, 3L, 3L, 3L,
3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L,
2L, 2L, 2L, 2L, 2L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L,
3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L), .Label = c("fp", "np", "tp"
), class = "factor"), Participant = c(21L, 22L, 23L, 24L, 25L,
26L, 27L, 28L, 29L, 30L, 21L, 22L, 23L, 24L, 25L, 26L, 27L, 28L,
29L, 30L, 21L, 22L, 23L, 24L, 25L, 26L, 27L, 28L, 29L, 30L, 21L,
22L, 23L, 24L, 25L, 26L, 27L, 28L, 29L, 30L, 21L, 22L, 23L, 24L,
25L, 26L, 27L, 28L, 29L, 30L, 21L, 22L, 23L, 24L, 25L, 26L, 27L,
28L, 29L, 30L, 21L, 22L, 23L, 24L, 25L, 26L, 27L, 28L, 29L, 30L,
21L, 22L, 23L, 24L, 25L, 26L, 27L, 28L, 29L, 30L, 21L, 22L, 23L,
24L, 25L, 26L, 27L, 28L, 29L, 30L, 21L, 22L, 23L, 24L, 25L, 26L,
27L, 28L, 29L, 30L, 21L, 22L, 23L, 24L, 25L, 26L, 27L, 28L, 29L,
30L, 21L, 22L, 23L, 24L, 25L, 26L, 27L, 28L, 29L, 30L, 21L, 22L,
23L, 24L, 25L, 26L, 27L, 28L, 29L, 30L, 21L, 22L, 23L, 24L, 25L,
26L, 27L, 28L, 29L, 30L, 21L, 22L, 23L, 24L, 25L, 26L, 27L, 28L,
29L, 30L, 21L, 22L, 23L, 24L, 25L, 26L, 27L, 28L, 29L, 30L, 21L,
22L, 23L, 24L, 25L, 26L, 27L, 28L, 29L, 30L, 21L, 22L, 23L, 24L,
25L, 26L, 27L, 28L, 29L, 30L, 21L, 22L, 23L, 24L, 25L, 26L, 27L,
28L, 29L, 30L, 21L, 22L, 23L, 24L, 25L, 26L, 27L, 28L, 29L, 30L,
21L, 22L, 23L, 24L, 25L, 26L, 27L, 28L, 29L, 30L, 21L, 22L, 23L,
24L, 25L, 26L, 27L, 28L, 29L, 30L, 21L, 22L, 23L, 24L, 25L, 26L,
27L, 28L, 29L, 30L, 21L, 22L, 23L, 24L, 25L, 26L, 27L, 28L, 29L,
30L, 21L, 22L, 23L, 24L, 25L, 26L, 27L, 28L, 29L, 30L, 21L, 22L,
23L, 24L, 25L, 26L, 27L, 28L, 29L, 30L, 21L, 22L, 23L, 24L, 25L,
26L, 27L, 28L, 29L, 30L, 21L, 22L, 23L, 24L, 25L, 26L, 27L, 28L,
29L, 30L, 21L, 22L, 23L, 24L, 25L, 26L, 27L, 28L, 29L, 30L, 21L,
22L, 23L, 24L, 25L, 26L, 27L, 28L, 29L, 30L), Accuracy = c(0.785398163397448,
0.523598775598299, 0.785398163397448, 0.523598775598299, 0.785398163397448,
0.869122203007293, 0.955316618124509, 0.785398163397448, 0.615479708670387,
0.701674123787604, 1.15026199151093, 1.15026199151093, 0.869122203007293,
0.523598775598299, 0.701674123787604, 0.701674123787604, 0.955316618124509,
0.701674123787604, 0.955316618124509, 0.615479708670387, 0.955316618124509,
0.785398163397448, 0.701674123787604, 0.869122203007293, 0.785398163397448,
0.615479708670387, 0.615479708670387, 0.869122203007293, 0.701674123787604,
0.615479708670387, 1.0471975511966, 0.869122203007293, 0.615479708670387,
0.615479708670387, 0.869122203007293, 0.701674123787604, 0.701674123787604,
0.869122203007293, 0.785398163397448, 0.869122203007293, 1.0471975511966,
0.955316618124509, 0.523598775598299, 1.0471975511966, 0.615479708670387,
0.955316618124509, 0.615479708670387, 0.785398163397448, 0.955316618124509,
0.785398163397448, 0.701674123787604, 0.615479708670387, 0.615479708670387,
0.955316618124509, 0.869122203007293, 0.869122203007293, 1.0471975511966,
0.785398163397448, 0.701674123787604, 0.785398163397448, 1.0471975511966,
0.911738290968488, 1.00028587904971, 0.827113206702756, 0.785398163397448,
1.00028587904971, 1.09681145610345, 1.00028587904971, 1.0471975511966,
1.09681145610345, 1.0471975511966, 0.827113206702756, 1.0471975511966,
0.420534335283965, 0.659058035826409, 1.0471975511966, 0.869122203007293,
1.0471975511966, 0.869122203007293, 0.785398163397448, 1.09681145610345,
0.785398163397448, 0.955316618124509, 0.911738290968488, 0.911738290968488,
1.00028587904971, 1.20942920288819, 1.15026199151093, 1.00028587904971,
1.20942920288819, 1.09681145610345, 1.0471975511966, 0.911738290968488,
0.827113206702756, 1.00028587904971, 0.969532110115768, 1.09681145610345,
1.00028587904971, 0.785398163397448, 1.09681145610345, 1.09681145610345,
0.869122203007293, 0.743683120092141, 0.869122203007293, 0.869122203007293,
1.0471975511966, 1.00028587904971, 1.09681145610345, 1.36522739563372,
1.00028587904971, 1.15026199151093, 0.869122203007293, 0.570510447745185,
1.20942920288819, 1.0471975511966, 0.955316618124509, 0.827113206702756,
1.00028587904971, 1.00028587904971, 1.0471975511966, 0.955316618124509,
0.911738290968488, 0.911738290968488, 0.570510447745185, 0.869122203007293,
1.00028587904971, 0.869122203007293, 0.785398163397448, 0.911738290968488,
0.869122203007293, 0.785398163397448, 0.701674123787604, 1.00028587904971,
0.420534335283965, 0.570510447745185, 0.969532110115768, 0.869122203007293,
0.911738290968488, 1.0471975511966, 0.785398163397448, 0.955316618124509,
0.827113206702756, 0.827113206702756, 0.659058035826409, 0.955316618124509,
0.701674123787604, 0.785398163397448, 0.785398163397448, 1.09681145610345,
1.0471975511966, 0.869122203007293, 0.827113206702756, 0.911738290968488,
0.827113206702756, 0.785398163397448, 0.827113206702756, 1.00028587904971,
0.911738290968488, 1.09681145610345, 0.955316618124509, 0.955316618124509,
1.15026199151093, 0.785398163397448, 0.955316618124509, 0.911738290968488,
1.0471975511966, 0.869122203007293, 0.869122203007293, 0.911738290968488,
0.955316618124509, 0.955316618124509, 0.827113206702756, 0.785398163397448,
0.869122203007293, 0.955316618124509, 0.684719203002283, 0.827113206702756,
1.00028587904971, 0.785398163397448, 0.827113206702756, 1.27795355506632,
1.20942920288819, 1.27795355506632, 1.00028587904971, 0.869122203007293,
1.15026199151093, 1.36522739563372, 1.27795355506632, 1.5707963267949,
1.5707963267949, 1.5707963267949, 1.27795355506632, 1.20942920288819,
0.911738290968488, 0.659058035826409, 1.36522739563372, 1.20942920288819,
1.36522739563372, 1.36522739563372, 1.27795355506632, 1.20942920288819,
1.0471975511966, 1.15026199151093, 1.15026199151093, 0.869122203007293,
1.27795355506632, 1.36522739563372, 1.27795355506632, 1.09681145610345,
1.36522739563372, 1.27795355506632, 1.00028587904971, 1.27795355506632,
1.15026199151093, 1.00028587904971, 1.36522739563372, 1.09681145610345,
1.15026199151093, 1.15026199151093, 1.36522739563372, 1.5707963267949,
1.5707963267949, 0.869122203007293, 1.09681145610345, 1.20942920288819,
1.36522739563372, 1.27795355506632, 1.27795355506632, 1.36522739563372,
1.5707963267949, 1.5707963267949, 1.15026199151093, 0.911738290968488,
1.20942920288819, 1.20942920288819, 1.28403977458335, 1.20942920288819,
1.36522739563372, 1.27795355506632, 1.36522739563372, 1.20942920288819,
0.911738290968488, 1.20942920288819, 1.0471975511966, 0.827113206702756,
1.5707963267949, 1.0471975511966, 1.0471975511966, 1.15026199151093,
1.27795355506632, 1.15026199151093, 1.00028587904971, 1.20942920288819,
0.659058035826409, 0.785398163397448, 1.09681145610345, 1.20942920288819,
0.827113206702756, 1.0471975511966, 1.20942920288819, 1.5707963267949,
0.955316618124509, 1.0471975511966, 1.0471975511966, 0.869122203007293,
1.20942920288819, 1.27795355506632, 1.09681145610345, 1.0471975511966,
1.5707963267949, 1.27795355506632, 0.869122203007293, 1.00028587904971,
0.911738290968488, 0.911738290968488, 1.00028587904971, 1.20942920288819,
1.20942920288819, 1.00028587904971, 1.36522739563372, 1.0471975511966,
1.09681145610345, 0.827113206702756, 1.15026199151093, 1.09681145610345,
1.27795355506632, 1.36522739563372, 1.36522739563372, 1.36522739563372,
1.15026199151093, 1.27795355506632, 0.955316618124509, 0.701674123787604,
1.09681145610345, 1.00028587904971, 1.20942920288819, 1.20942920288819,
1.20942920288819, 1.00028587904971, 1.36522739563372)), .Names = c("Coherence",
"PrimeDuration", "PrimeType", "Participant", "Accuracy"), row.names = c(20L,
21L, 22L, 23L, 24L, 25L, 26L, 27L, 28L, 29L, 49L, 50L, 51L, 52L,
53L, 54L, 55L, 56L, 57L, 58L, 78L, 79L, 80L, 81L, 82L, 83L, 84L,
85L, 86L, 87L, 107L, 108L, 109L, 110L, 111L, 112L, 113L, 114L,
115L, 116L, 136L, 137L, 138L, 139L, 140L, 141L, 142L, 143L, 144L,
145L, 165L, 166L, 167L, 168L, 169L, 170L, 171L, 172L, 173L, 174L,
194L, 195L, 196L, 197L, 198L, 199L, 200L, 201L, 202L, 203L, 223L,
224L, 225L, 226L, 227L, 228L, 229L, 230L, 231L, 232L, 252L, 253L,
254L, 255L, 256L, 257L, 258L, 259L, 260L, 261L, 281L, 282L, 283L,
284L, 285L, 286L, 287L, 288L, 289L, 290L, 310L, 311L, 312L, 313L,
314L, 315L, 316L, 317L, 318L, 319L, 339L, 340L, 341L, 342L, 343L,
344L, 345L, 346L, 347L, 348L, 368L, 369L, 370L, 371L, 372L, 373L,
374L, 375L, 376L, 377L, 397L, 398L, 399L, 400L, 401L, 402L, 403L,
404L, 405L, 406L, 426L, 427L, 428L, 429L, 430L, 431L, 432L, 433L,
434L, 435L, 455L, 456L, 457L, 458L, 459L, 460L, 461L, 462L, 463L,
464L, 484L, 485L, 486L, 487L, 488L, 489L, 490L, 491L, 492L, 493L,
513L, 514L, 515L, 516L, 517L, 518L, 519L, 520L, 521L, 522L, 542L,
543L, 544L, 545L, 546L, 547L, 548L, 549L, 550L, 551L, 571L, 572L,
573L, 574L, 575L, 576L, 577L, 578L, 579L, 580L, 600L, 601L, 602L,
603L, 604L, 605L, 606L, 607L, 608L, 609L, 629L, 630L, 631L, 632L,
633L, 634L, 635L, 636L, 637L, 638L, 658L, 659L, 660L, 661L, 662L,
663L, 664L, 665L, 666L, 667L, 687L, 688L, 689L, 690L, 691L, 692L,
693L, 694L, 695L, 696L, 716L, 717L, 718L, 719L, 720L, 721L, 722L,
723L, 724L, 725L, 745L, 746L, 747L, 748L, 749L, 750L, 751L, 752L,
753L, 754L, 774L, 775L, 776L, 777L, 778L, 779L, 780L, 781L, 782L,
783L, 803L, 804L, 805L, 806L, 807L, 808L, 809L, 810L, 811L, 812L,
832L, 833L, 834L, 835L, 836L, 837L, 838L, 839L, 840L, 841L, 861L,
862L, 863L, 864L, 865L, 866L, 867L, 868L, 869L, 870L), class = "data.frame")

Here's the code for ezANOVA Type III:

Cond_Per_Row_stats$Participant <- as.factor(Cond_Per_Row_stats$Participant)
Exp1Model <- ezANOVA(data = Cond_Per_Row_stats, dv = .(Accuracy), wid = .(Participant), within = .(Coherence, PrimeDuration, PrimeType), type=3, detailed = TRUE)

#Exp1Model
Exp1Model$ANOVA

And here is the ezANOVA output:

                            Effect DFn DFd          SSn        SSd            F            p p<.05         ges
1                       (Intercept)   1   9 297.26560885 2.04482984 1308.3682683 4.670956e-11     * 0.973889040
2                         Coherence   4  36   7.79974598 1.05880149   66.2992210 4.175417e-16     * 0.494602047
3                     PrimeDuration   1   9   0.10509137 0.08667666   10.9120771 9.180827e-03     * 0.013014273
4                         PrimeType   2  18   0.13699186 1.61652086    0.7627039 4.808964e-01       0.016898001
5           Coherence:PrimeDuration   4  36   0.13548043 0.49918172    2.4426453 6.426183e-02       0.016714681
6               Coherence:PrimeType   8  72   0.14385084 1.19426233    1.0840646 3.841289e-01       0.017729058
7           PrimeDuration:PrimeType   2  18   0.02961932 0.47325423    0.5632784 5.790571e-01       0.003702594
8 Coherence:PrimeDuration:PrimeType   8  72   0.09786368 0.99646749    0.8838955 5.343856e-01       0.012130069

When I use aov() with advised code to get Type III output:

Cond_Per_Row_stats$Participant <- as.factor(Cond_Per_Row_stats$Participant)
        options(contrasts=c("contr.sum", "contr.poly"))
    aovModel <- aov(Accuracy ~ Coherence * PrimeDuration * PrimeType, data = Cond_Per_Row_stats)
    ## drop1() supposedly spits out Type III results...
    drop1(aovModel,~.,test="F")

I get this output:

    Model:
Accuracy ~ Coherence * PrimeDuration * PrimeType
                                  Df Sum of Sq     RSS      AIC F value  Pr(>F)    
<none>                                          7.9700 -1028.43                    
Coherence                          4    7.7997 15.7697  -831.71 66.0581 < 2e-16 ***
PrimeDuration                      1    0.1051  8.0751 -1026.50  3.5602 0.06025 .  
PrimeType                          2    0.1370  8.1070 -1027.32  2.3204 0.10019    
Coherence:PrimeDuration            4    0.1355  8.1055 -1031.37  1.1474 0.33457    
Coherence:PrimeType                8    0.1439  8.1138 -1039.06  0.6092 0.76998    
PrimeDuration:PrimeType            2    0.0296  7.9996 -1031.32  0.5017 0.60606    
Coherence:PrimeDuration:PrimeType  8    0.0979  8.0679 -1040.77  0.4144 0.91186

As can be seen the outputs for F-values and the corresponding p-values differ quite a bit (apart from coincidental similarity in 'Coherence' factor).

Firstly, does anyone know why the outputs are different considering I followed advice on how to change aov() to Type III output? The sum of squares match but shouldn't everything else also?

Secondly, can anyone suggest how I can ensure aov() outputs the same F-values as ezANOVA?

I've exhausted all avenues that I can find to get to this point. Any advice would be very welcome.

$\endgroup$
6
$\begingroup$

The reason why you are getting different results, is that you have a within subjects / repeated measures design (at least that is what I concluded from the used ezANOVA function). The way you specified your aov() model uses the predictors as you should do in a between subjects design (with interactions because you used * instead of +).

When you want to specify it as a within subjects design with aov(), you can specify this as follows:

m.aov <- aov(Accuracy ~ (Coherence * PrimeDuration * PrimeType) + 
               Error(Participant/(Coherence * PrimeDuration * PrimeType)), 
            data = Cond_Per_Row_stats)

Now let's compare that to the ezANOVA approach:

library(ez)
m.ez <- ezANOVA(data = Cond_Per_Row_stats, 
                dv = .(Accuracy), 
                wid = .(Participant), 
                within = .(Coherence, PrimeDuration, PrimeType), 
                type = 3, detailed = TRUE)

Looking at the output of summary(m.aov):

> summary(m.aov)

Error: Participant
          Df Sum Sq Mean Sq F value Pr(>F)
Residuals  9  2.045  0.2272               

Error: Participant:Coherence
          Df Sum Sq Mean Sq F value   Pr(>F)    
Coherence  4  7.800  1.9499    66.3 4.18e-16 ***
Residuals 36  1.059  0.0294                     
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Error: Participant:PrimeDuration
              Df  Sum Sq Mean Sq F value  Pr(>F)   
PrimeDuration  1 0.10509 0.10509   10.91 0.00918 **
Residuals      9 0.08668 0.00963                   
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Error: Participant:PrimeType
          Df Sum Sq Mean Sq F value Pr(>F)
PrimeType  2  0.137 0.06850   0.763  0.481
Residuals 18  1.617 0.08981               

Error: Participant:Coherence:PrimeDuration
                        Df Sum Sq Mean Sq F value Pr(>F)  
Coherence:PrimeDuration  4 0.1355 0.03387   2.443 0.0643 .
Residuals               36 0.4992 0.01387                 
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Error: Participant:Coherence:PrimeType
                    Df Sum Sq Mean Sq F value Pr(>F)
Coherence:PrimeType  8 0.1439 0.01798   1.084  0.384
Residuals           72 1.1943 0.01659               

Error: Participant:PrimeDuration:PrimeType
                        Df Sum Sq Mean Sq F value Pr(>F)
PrimeDuration:PrimeType  2 0.0296 0.01481   0.563  0.579
Residuals               18 0.4733 0.02629               

Error: Participant:Coherence:PrimeDuration:PrimeType
                                  Df Sum Sq Mean Sq F value Pr(>F)
Coherence:PrimeDuration:PrimeType  8 0.0979 0.01223   0.884  0.534
Residuals                         72 0.9965 0.01384

you see that you now get the same result compared to the output of m.ez$ANOVA:

                             Effect DFn DFd          SSn        SSd            F            p p<.05         ges
1                       (Intercept)   1   9 297.26560885 2.04482984 1308.3682683 4.670956e-11     * 0.973889040
2                         Coherence   4  36   7.79974598 1.05880149   66.2992210 4.175417e-16     * 0.494602047
3                     PrimeDuration   1   9   0.10509137 0.08667666   10.9120771 9.180827e-03     * 0.013014273
4                         PrimeType   2  18   0.13699186 1.61652086    0.7627039 4.808964e-01       0.016898001
5           Coherence:PrimeDuration   4  36   0.13548043 0.49918172    2.4426453 6.426183e-02       0.016714681
6               Coherence:PrimeType   8  72   0.14385084 1.19426233    1.0840646 3.841289e-01       0.017729058
7           PrimeDuration:PrimeType   2  18   0.02961932 0.47325423    0.5632784 5.790571e-01       0.003702594
8 Coherence:PrimeDuration:PrimeType   8  72   0.09786368 0.99646749    0.8838955 5.343856e-01       0.012130069
| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ That looks good so far. I'll check it against my larger data set and select as answered once I've double checked. Thank you very much for that! $\endgroup$ – Docconcoct Mar 10 '16 at 15:43
  • 1
    $\begingroup$ This is the correct answer. $\endgroup$ – Henrik Mar 10 '16 at 16:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.