One way ANOVA or one way repeated measures ANOVA?

Question

If I measure students' output performances on the same activity with different time intervals, i.e. 1 hour, 40 minutes and 20 minutes and want to know if there is a significant difference between those output performances, will I use one-way ANOVA or one way repeated measures ANOVA (since the same subject undergoes the same activity in three different time intervals)? But I'm confused since each output performance has nothing to do with each other.

Answer 1

Fake data for example. Here is an example with fake data. Suppose you have $n = 10$ students, each tested at times 20, 40, and 60 minutes.

               Time
        -------------------   
Student    20     40     60
     1  70.16  70.06  74.84
     2  77.06  83.13  89.89
     3  69.05  73.01  79.46
     4  86.02  87.90  92.61
     5  64.88  65.13  67.17
     6  69.72  68.62  70.60
     7  85.59  83.73  86.94
     8  87.60  92.48  94.24
     9  84.10  88.43  91.28
    10  81.88  85.59  86.63

Correct analysis as block design. Then a correct analysis takes both factors Time and Student into account. This is sometimes called a 'block' design (Students as blocks), sometimes called a 'two-way ANOVA with one replication per cell', and sometimes called a repeated measures design.

The resulting ANOVA table is as follows:

Source    DF       SS      MS      F      P
Time       2   168.95   84.48  19.20  0.000
Student    9  2256.38  250.71  56.97  0.000
Error     18    79.22    4.40
Total     29  2504.54

Notice that both Time and Student effects are highly significant. (Here P-value 0.000 means a value smaller than 0.0005.) I guess you assume students will perform differently because they have diverse abilities. (If you had several measurements on each student at each time, then you would declare 'Student' as a random effect, but with only one observation per 'cell' that distinction would not affect the ANOVA table.)

Incorrect analysis as one-way ANOVA. If you were to perform a one-factor (one-way) ANOVA (sometimes called a 'completely randomized design'), pretending that there were 30 different students in the experiment (10 at each time point), then you get the following ANOVA table.

Source  DF      SS      MS  F-Value  P-Value
Time     2   169.0   84.48     0.98    0.390
Error   27  2335.6   86.50
Total   29  2504.5

We can detect no differences among the three times here. So, even if you don't feel it is important to test whether Students differ, using Students as a second factor is important to getting a correct result. Notice that SS(Time) and MS(Time) are the same for the two models. However, in the correct block design, the denominator for F(Time) is MS(Error) = 4.04, so that F(Time) is large enough to be significant when compared with the F-distribution with 2 & 18 degrees of freedom.

Distinctions between designs. By contrast, in the incorrect one-way ANOVA design, the denominator for F(Time) is MS(Error) = 86.50, leading to a much smaller F(Time) and no significant result. A key difference between the the two ANOVA tables is that the block design has a smaller SS(Error) because 2256.38 has been 'taken from the Error line and absorbed by the Student line'.

In effect in the block design, every individual student is their own sub-experiment. Differences in scores between times are judged on a student-by-student basis. In the one-way ANOVA, each student at time 20 is less-precisely judged against the overall performance of 20 others at the other two times.

In terms of the data, notice that the three 10-vectors for different times are highly correlated: for example, $r_{20,40} = 0.960,\, r_{40,60} = 0.978.$ (In data for a one-way ANOVA there should be no significant correlations among columns, because there would be different randomly and independently chosen students in each column.)