Interpreting 5-way Mixed Model ANOVA

Question

I'm running a 2x2x2x2x2 mixed model ANOVA (on SPSS v21) for my study and found two 4-way interactions, one 5-way interactions and a couple of 3-way interactions. Whilst I understand how to interpret a 3-way ANOVA. I'm having quite a hard time trying to interpret the 4/5-way interactions.

My variables are:

• Within-subject factors: lineup sex (female, male), lineup ethnicity (Asian, Caucasian)
• Between-subject factors: lineup procedure (sequential, simultaneous), participant ethnicity (Asian, Caucasian), participant sex (female, male)

Sample size: 552

I've gotten to the stage where I have done separate ANOVAs (splitting the data) on each factor following a 5-way or 4-way interactions obtained from the initial analysis (e.g., AxB at C1, AxB at C2 and so on) and obtained a ton of output. Some of the output had only significant main effect while others had no sig. effects. But the graph that the SPSS produced with the estimated marginal means clearly indicates a significant interaction.

E.g: Lineup ethnicity x Participant sex at two levels of Lineup Sex.

Male Caucasians Lineup: Lineup ethnicity x Participant Sex

• All main effects & interaction are non. sig.

Female Caucasian Lineup: Lineup ethnicity x Participant Sex

• All main effects & interaction are non. sig.

But the graph indicates an interaction effect only for Female Caucasians.

I've consulted with my supervisor about this and due to the time constraints, he has advised me to just compare and contrast the graphs of each ANOVA. That much I understand but I am clueless as to what to do next. Yes, I compare between the graphs but am unsure on how that would help explain the 5/4-way interactions..

Any help would be greatly appreciated!

Answer 1

Generally, you should start from the highest order interactions. You are probably aware that it is usually not sensible to interpret a main effect A when that effect is also involved in an interaction A:B. This is because the interaction tells you that the effect of A actually depends on the level of B, rendering any simple main effect interpretation of A impossible. In the same way, if you have factors A, B, C, then A:B should not be interpreted if A:B:C is significant.

Thus, when you have a 5-way interaction, none of the lower-order interactions can be sensibly interpreted. Therefore, if I understand you correctly and you have interpreted your lower order interactions, you should probably not continue along those lines.

Rather, what you can do is to split up your data set and continue to analyze factor levels of your data set separately. Which of the factors you use to split up the dataset is arbitrary, but often it is very useful to split up the data for each variable and assess what you see. In your example, you might start with sex, and calculate an ANOVA for males, and another one for females (each ANOVA contains the 4 remaining factors). Just as well, you could split up the data according to ethnicity (one ANOVA for Asian, one for Caucasian). You could also split up by one of the within-subject factors.

I will assume that you have decided to split the data by sex (just to continue with the example here). Then, assume that for males, you get a 4-way interaction. You would then go on to split up the male data by one of the remaining variables (say, ethnicity). You would then calculate ANOVAs for male Asians (over the remaining 3 factors), and for male Caucasians.

Importantly, if you get only a lower-order interaction, then you are only "allowed" to analyze these further. This is because the other factors did not show significant differences. Thus, if your males ANOVA gives you only a 2-way interaction, then you would average over the other factors and calculate only an ANOVA over the 2 interacting factors (and, because we are in the male part of the ANOVAs, this would be for the males alone).

For the females, everything may look different, and so the decision which follow-up ANOVAs to calculate is separate for this group. So, what you did for males should be done for females in the same way ONLY if you got the same interactions.

Thus, you will potentially have a lot of ANOVAs, and it might not be easy to decide which ones to report. You should report 1 complete line down from the hightest interaction to the last effects (possibly t-tests to compare only 1 of your factors at the end). You should not usually report several lines (e.g., one starting the split-up by sex, then another one starting by ethnicity). However, you must report a complete line, and cannot simply choose to report only some of the ANOVAs of that line. So, you report one complete analysis, not more, not less. Which way to go in terms of splitting up / follow-up ANOVA is a subjective decision (unless you have clear hypotheses you can follow), and might depend on which results can be understood best etc.