Binary response outcome DV with two crossed repeated measure factors and one between subject factor. Should I use Generalized Linear Mixed Model?

Question

I have a complex experimental design with two within-subject factors and one between-subject factor, where my outcome variable is measured on a binary response.

1. Country (Between-subject, 2 levels: A and B)

2. Experiment (Within-subject, 5 levels: 1,2,3,4)

3. Trial (Within-subject, 3 levels: 1,2,3)

4. DV/outcome measure is a binary response (success or failure).

This means all the subjects go through all possible 15 conditions (5 Experiments x 3 Trials).

I want to know:

If there is a Country group difference in Trials across different Experiments, in other words, if there is a three-way interaction.

If there is an interaction, and I want to explore pairwise comparisons (e.g., is Country A more likely to succeed in Trial 1 in Experiment 1 than Country B?). I do not have specific hypotheses about which specific pairs would be different, however.

This would have been easier to test using three-way mixed-factorial ANOVA if my DV is a continuous variable, but my understanding is I won't be able to test in parametric procedures including ANOVAs. Would Generalized Linear Mixed Modeling (GLMM) with a binomial distribution option be the appropriate procedure to pursue and if so, how do I write this out in lme4 function in R?

My understanding is that Subject would be a random effect, and Country would be a fixed effect, but not sure about Experiment and Trial variables.

I am also a bit confused with which variable is crossed vs. nested. Do I have any nesting variables?

I am in Psychology and very new to multilevel modeling and mixed-effects modeling, so I would greatly appreciate your help and the simplest way to test it.

Answer 1

It seems to me that a multilevel logistic regression model would work well. You can use glmer function to do that.

Your experiment and trial, if considered as random effects, seem to be crossed (i.e. trials were equivalent across experiments, e.g. trial 1 in experiment 1 was similar to trial 1 in experiment 2 etc. If trials were somehow experiment-specific, then experiment and trial would be nested).

However, in your case, I probably wouldn't put experiment and trial in as random effects, because it makes more sense to use them as factorial fixed effects as you are interested in possible differences between different experiments and trials.

A possible model for your situation would go

model<-glmer(outcome ~ (1|id)+Country*Experiment*trial, data=data, family="binomial") #id=participant signifier

which gives you the main effects and all interactions between country, experiment, and trial, while controlling for participant-specific variance in outcome. You'd need to specify factor(Experiment) and factor(trial) if experiment and trial are numeric in your data. You can then investigate the specific contrasts further e.g. with emmeans package functions.