Multilevel between-subject design using glmer I want to test the effect of different learning strategies across time. 


*

*Conditions were manipulated between subjects. 

*Each participant answered the same set of binary questions across 3-time points. 


 


*

*Condition = Nested (Between)

*Time = Crossed (Within)

*Questions = Crossed (Within)


When I graphed the data (mean for each condition) I can see that all conditions are equal at time0 (pretest) then develop (learn) differently across time1 and time2. 
I set up my model like this:
model <- glmer(Outcome ~ Condition*Time + (Time|Question/ID), 
               data = learn, 
               family = binomial(link = "logit"), 
               control = glmerControl(optimizer = c("bobyqa")),
               nAGQ = 1)



*

*Am I moving in the right direction?

*How would I control for how participants answered on the pretest (time0)?
 A: It seems that you have indeed a crossed design. In particular, you would expect that answers from the same subjects are correlated, and that answers in the same questions will be correlated. In this case you could start first from the random intercepts model:
fm1 <- glmer(Outcome ~ Condition * Time + 
                (1 | ID) + (1 | Question), 
             data = learn, family = binomial())

You could then try adding random slopes for each grouping factor to see it improves the fit, i.e.,
fm21 <- glmer(Outcome ~ Condition * Time + 
                (Time | ID) + (1 | Question), 
              data = learn, family = binomial())

anova(fm1, fm21)

and
fm22 <- glmer(Outcome ~ Condition * Time + 
                (1 | ID) + (Time | Question), 
              data = learn, family = binomial())

anova(fm1, fm22)

And perhaps whether adding slopes for both improves the fit, i.e.,
fm3 <- glmer(Outcome ~ Condition * Time + 
                (Time | ID) + (Time | Question), 
              data = learn, family = binomial())

anova(fm21, fm3)
anova(fm22, fm3)

A: Thanks for the excellent reply. I ran all the models and the following anova's to compare the models and the fm3 model was the best fit. I then moved on comparing fm3 to my original model, output:
Data: learn
Models:
fm3: Outcome ~ Condition * Time + (Time | ID) + (Time | Question)
modelx2: Outcome ~ Condition * Time + (Time | Question/ID)
        Df  AIC  BIC logLik deviance Chisq Chi Df          Pr(>Chisq)    
fm3     24 9524 9695  -4738     9476                                     
modelx2 24 8618 8789  -4285     8570 905.8      0 <0.0000000000000002 ***



*

*Conceptually, can I draw the conclusion that difficulty (Outcome) for different questions varies within different questions and for different individuals? 

*Would there be any reason to add a random effect for condition, since participants would be nested within condition across time? 

*If I would like to control for answers on the pretest (time0) could I "break out" time0 and include it as a predictor? For example, the effect of conditions may vary due to how good participants were from the beginning. 
