I'm sorry for such a silly question, but I would like to ask about post hoc comparisons that I could use to follow up on a significant, although hard to interpret, three-way interaction.
I ran a mixed-effects logistic regression model with glmer() predicting categorization responses from the speech cue weighting task with the following formula:
response ~ Time * Group * pitch_level * duration_level
where Time has 3 levels (Time I, Time II, Time III) coded with contr.sdif(3) to compare subsequent differences between Time I vs Time II, and then Time II vs Time III, Group has 2 levels (experimental or control), and pitch_level and duration_level both have 4 levels of how much of pitch or duration information was there in a given trial.
I have a significant 3-way interaction of Time (I-II) * Group * pitch_level that suggests some differences from Time I to II between the groups in how much they rely on pitch while categorizing speech, but to understand which group uses more pitch than the other and at which testing time, we need to run some post hoc analyses. Is that correct?
What post hoc analyses do you think would be appropriate here?
I can think of two options: (1) One is to break down the model by Group to see if the significant effect of time is still there - but this would be effectively another analysis which does not take into account all the dataset but only its subsets. Isn't problematic? How would I interpret significant two-way interactions of Time x pitch_level in each group then? (2) Second, to compare performance across Time x Group x pitch levels with emmeans() across all the pitch levels, but then we do not take into account the interaction effects since these are just pairwise comparisons.
Do you think any of these make sense? Or is there anything obvious I'm missing here?
I would very much appreciate your advice, thanks!