I just started reading about multilevel modeling so I can apply the method to my research data. However, I have yet to come across an illustrative example or even discussion how to organize and set up a model for a repeated-measures design with a grouping variable to counterbalance across materials.

For example, let’s say I want to look at the effect difficulty level of paragraph has on time to read a target sentence embedded within each paragraph. To test this, I create 6 paragraphs, in which there are three versions of each paragraph: Difficult (D), Moderate (M), Easy (E). Participants get exposed to all three levels of difficulty but only read one version of each paragraph. So, reading material is repeated-measures, but the specific paragraph participants get differs. To counterbalance across difficulty levels, participants also get different orders of conditions with the restriction that each condition appears in each position an equal number of times (this is the grouping variable; order is a between subjects variable)

So, for the sake of counterbalancing, I administer three different orders of the six paragraphs for the following participants: Beth: DME EDM (order 1) John: MED DME (order 2) Mary: EDM MED (order 3)

So, you can see that everyone gets all levels of difficulties but for different paragraphs.

The dependent measure is the time to read a target sentence for each paragraph. Here is a tabular display of fictitious data of reading time laid out by the experimental design:

enter image description here

So, my first question is, how do I organize the data to prepare for MLM to reflect the order grouping variable and three levels of difficulty, repeated measures? Then how should I start thinking about incorporating random effects into the model?


1 Answer 1


The way you have presented the data in the table is a start but needs some additions and changes in order to analyze the data using a mixed modeling framework. The most important thing is that each subject has multiple rows corresponding to a repeated measure. The time to read measure needs to be in its own column. Then you want to create categorical variables representing each of the conditions: version with three categories, and potentially order with three categories (assuming there are only three of them). Your dataset would look something like this:

subject read_time version order 
1        1800        1      1 
1        1500        1      1 
1        1200        2      1
1        1400        2      1
1        1100        3      1
1        1000        3      1 
2        1600        1      2 
2        1550        1      2
2        1600        2      2
2        1350        2      2
2        1000        3      2
2        1100        3      2 

And so on. If I understand your design, version changes within subjects (although all subjects are exposed to all versions) whereas order is a between-subjects factor that is constant within subject.

In terms of modeling this with a mixed model, that depends on your research questions, but the basic model I would set up would regress read_time on the fixed categorical predictors version and order with a random intercept for subject given that we expect responses from the same subject to be more highly correlated than responses from different subjects. In R, you can set up the model using lme4 as such:

base <- lmer(read_time ~ version + order + (1|subject), data=df)

This fixed effect coefficients from this model will give test the read_time contrasts between the base category for version (presumably 1) against version 2 and version 3 and then provide a set of read_time contrasts between the base category of order (presumably 1) against order 2 and order 3.

You can get the ANOVA F test for this model if you want using anova(base). You may want to subsequently test whether version 2 is different from version 3 and likewise whether order 2 is different from order 3. You can use the multcomp package in R for this. See https://stats.idre.ucla.edu/r/faq/how-can-i-test-contrasts-in-r/


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.