How to analyze nested data of repeated measurements? ANCOVA or linear mixed model?

Question

I have obtained a dataset from a psychological experiment.

In the experiment, each subject completed four sessions of tasks on four different days. Each session consisted of two stages of tasks. During the first stage, the subjects completed the tasks without any additional stimuli. In the second stage, the subjects listened to music while performing the tasks. Each stage had three blocks of tasks. The music played on different days was different.

Dependent variable: Performance

Another variables: Stage(stage1,stage2), types of Music(A,B,C,D), subjects Id (1:27)

I am interested in analyzing how music influenced the subjects' performance during the tasks.From my point of view, an ANCOVA or a linear mixed model seems to be proper. I have performed

1. ANCOVA with performance of stage 1 as the Covariates, performance of stage 2 as the dependent variable.

   aov(Stage2 ~ Stage1 * Music + Error(Id))

2. linear mixed model with performance as the dependent variable, stage and music as the independent variable, subject id as the random term.

   lmer(Performance ~ Stage * Music + (1|Id)

However, I feel like that isn't correct. I noticed that the model structure should have music nested within stage (Because music only played during stage 2). So that I tried

3. linear mixed model with performance as the dependent variable, stage and music as the independent variable and have an nested structure, subject id as the random term.

   lmer(Performance ~ Stage + Stage:Music + (1|Id)

Question: What are the differences between these models? Which one is better?

Answer 1

This is an interesting designed experiment.

First I'll illustrate my understanding the design with a data frame slice. Here are the rows corresponding to Subject 1 and Repeat 1. (Aside: The question refers to "three blocks of tasks" but in a comment the OP clarifies that the same task is performed three times; I think the term "repeat" suits the design better.)

design %>% filter(Id == 1, Rep == 1)
#> A tibble: 8 × 5
#>      Id Session Stage Music   Rep
#>   <int> <chr>   <chr> <chr> <int>
#> 1     1 1       1     0         1
#> 2     1 1       2     D         1
#> 3     1 2       1     0         1
#> 4     1 2       2     B         1
#> 5     1 3       1     0         1
#> 6     1 3       2     C         1
#> 7     1 4       1     0         1
#> 8     1 4       2     A         1

There are 4 sessions (on a different day) and two stages (always in the order "1" then "2"). The no music setting (Music = 0) corresponds to the early stage (Stage = 1). There are 4 types of music ("A", "B", "C", "D"); a different type of music was played during the late stage (Stage = 2) of each session and music types were assigned to sessions in a random order.

We can see from the design that Stage is confounded with Music since Stage = 1 whenever Music = 0. So once the model includes a fixed Music effect, it's redundant to include Stage.

This is something of a weakness of the design as we have to assume there is no Stage effect in order to argue that the Music effect estimates are unbiased. (If there were Stage effects we can't separate them from the Music effects.) This assumption may be perfectly reasonable depending on the nature of the task; in any case, the assumption should be stated clearly.

And what about random effects? It's natural of course to include random subject effects but the design has more structure: performances clustered within session. For example, a participant might not have gotten enough sleep the night before a session, so they perform a bit worse (in both stages on that day). This can be captured with a nested random effect: (1 | Id / Session).

Furthermore, I make the assumption that there is no trend for the subjects to get better at performing the task over time, ie. in the course of the four sessions. Whether this is a reasonable assumption again depends on the nature of the task and perhaps on the time between sessions. (Was there enough time for subjects to improve with practice?)

Putting these arguments together we get:

Performance ~ Music + (1 | Id / Session)

---

PS. Unlike the "no Stage effect" assumption, the "no Session effect" assumption can be relaxed because the design allows us to estimate Session effects. In that case, I would fit the following model with random Session effects:

Performance ~ Music + Session + (Session | Id)

where Session is a categorical variable (factor), not a numeric variable.

The nature of the experiment and the task should determine which model is more appropriate. In short, it depends on whether there might be a trend in time or not.