5
$\begingroup$

In a balanced repeated measures one-way ANOVA, we compare the true averages of a response variable across time points. Are the results about the time effects equivalent to those from a classic two-way ANOVA with "time" and "subject id" as factors? If yes, do you know a good reference about it?

Example in R (20 subject ids with one value for each of the three time points):

# Data generation
set.seed(2)
t0 <- rnorm(20)
t1 <- rexp(20) + t0 / 4
t2 <- runif(20) + t1 / 2
response <- c(t0, t1, t2)
time <- factor(rep(1:3, each = 20))
id <- factor(rep(1:20, times = 3))
#___________________________________________________________

# Two-way between-subject ANOVA
drop1(lm(response ~ time + id), test = "F")

# Output -> p-value of any time effect is 0.0013734 

       Df Sum of Sq    RSS      AIC F value    Pr(>F)    
<none>              21.038 -18.8811                      
time    2     8.723 29.761  -2.0690  7.8780 0.0013734 ** 
id     19    39.129 60.167   6.1669  3.7199 0.0002787 ***
#___________________________________________________________

# Now the repeated-measures ANOVA
summary(aov(response ~ time + Error(id)))

# Output -> p-value also 0.00137
Error: id
          Df Sum Sq Mean Sq F value Pr(>F)
Residuals 19  39.13   2.059               

Error: Within
          Df Sum Sq Mean Sq F value  Pr(>F)   
time       2  8.723   4.361   7.878 0.00137 **
Residuals 38 21.038   0.554                   

As far as I know, the methods provide also similar (wrong) p-values in the unbalanced case.

$\endgroup$
  • 1
    $\begingroup$ A similar question stats.stackexchange.com/q/17378/3277. $\endgroup$ – ttnphns Mar 6 '15 at 18:27
  • $\begingroup$ Thx, that is very helpful. $\endgroup$ – Michael M Mar 6 '15 at 19:06
  • $\begingroup$ (+1) Note that this equivalence only holds if you have exactly 1 data points per subject-time combination. If for each of the 20 subjects for each of the 3 time points you have $k>1$ measurements, then RM-ANOVA will not be equivalent to two-way ANOVA (the SS decomposition will be the same, but the F-values and p-values for the time effect will differ). $\endgroup$ – amoeba says Reinstate Monica Jun 18 '17 at 22:04
  • $\begingroup$ Follow-up question based on my above comment: stats.stackexchange.com/questions/286280. $\endgroup$ – amoeba says Reinstate Monica Jun 20 '17 at 10:03
5
$\begingroup$

Well ... yes. Your scenario is actually very simple. It's basically a one-way randomized block design, where the blocks are individuals and Time is the treatment. The error for estimating the Time effect is the id x Time interaction with 38 degrees of freedom. You are treating id as a fixed (not random) effect, so you get the same answer either way, since you're doing the same thing.

The full beauty of repeated measures designs kicks in when you have within and between subject effects. Suppose 10 of the id's had treatment A and 10 had treatment B. Then the error for the A/B test is the id effect (with 19 DF's), and the error for the time effect is the "within subject error" with 38 DF's. The sums of squares for the effects themselves remain the same as they would be in a plain Jane two-way design; it's the denominator of the "F" tests that changes.

I observe that Douglas Montgomery's "design and analysis of experiments" is still in print and currently running its 8th edition. I recommend an old fashioned textbook for this kind of question, where the impact of various designs is explained in depth. Another interesting old-fashioned text is "The analysis of messy data" by Milliken and Johnson, which marches you through the difference between balanced and unbalanced designs, one expected mean square after another.

$\endgroup$
  • $\begingroup$ Excellent hints. I was checking the p value of a time/treatment interaction too and the two approaches where also similar. $\endgroup$ – Michael M Mar 6 '15 at 19:12
  • $\begingroup$ (+1) But I would note that this equivalence (RM-ANOVA with 1 RM factor time = 2-way ANOVA with subject*time) only holds if one has exactly 1 data point per subject-time combination. If there are more measurements per subject/time, then RM-ANOVA will not be equivalent to two-way ANOVA: the SS decomposition will be the same, but the F-values and p-values for the time effect will differ. Normal ANOVA compares time variance to the residual variance, whereas RM-ANOVA compares time variance to the time-subject interaction variance. I actually don't fully understand why; can you comment on that? $\endgroup$ – amoeba says Reinstate Monica Jun 19 '17 at 7:30
  • 1
    $\begingroup$ Following up on my above comment, I have posted a new question: stats.stackexchange.com/questions/286280. Would be very grateful for any input! Cheers. $\endgroup$ – amoeba says Reinstate Monica Jun 20 '17 at 10:03
  • $\begingroup$ @amoeba: thx a lot for your inputs (+1) to your new question! $\endgroup$ – Michael M Jun 20 '17 at 12:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.