4
$\begingroup$

I'm running a 2x2x2x2x2 mixed model ANOVA (on SPSS v21) for my study and found two 4-way interactions, one 5-way interactions and a couple of 3-way interactions. Whilst I understand how to interpret a 3-way ANOVA. I'm having quite a hard time trying to interpret the 4/5-way interactions.

My variables are:

  • Within-subject factors: lineup sex (female, male), lineup ethnicity (Asian, Caucasian)
  • Between-subject factors: lineup procedure (sequential, simultaneous), participant ethnicity (Asian, Caucasian), participant sex (female, male)

Sample size: 552

I've gotten to the stage where I have done separate ANOVAs (splitting the data) on each factor following a 5-way or 4-way interactions obtained from the initial analysis (e.g., AxB at C1, AxB at C2 and so on) and obtained a ton of output. Some of the output had only significant main effect while others had no sig. effects. But the graph that the SPSS produced with the estimated marginal means clearly indicates a significant interaction.

E.g: Lineup ethnicity x Participant sex at two levels of Lineup Sex.

Male Caucasians Lineup: Lineup ethnicity x Participant Sex

  • All main effects & interaction are non. sig.

Female Caucasian Lineup: Lineup ethnicity x Participant Sex

  • All main effects & interaction are non. sig.

But the graph indicates an interaction effect only for Female Caucasians.

I've consulted with my supervisor about this and due to the time constraints, he has advised me to just compare and contrast the graphs of each ANOVA. That much I understand but I am clueless as to what to do next. Yes, I compare between the graphs but am unsure on how that would help explain the 5/4-way interactions..

Any help would be greatly appreciated!

$\endgroup$
  • 4
    $\begingroup$ Given that you have 5 dichotomous variables, I am not sure what graphs you created. Can you tell us? Also, even though it's a 5 way interaction, it's only $2^5 = 32$ combinations, so you could look at the predicted value for every combination without it being too overwhelming. $\endgroup$ – Peter Flom Oct 17 '13 at 10:17
  • $\begingroup$ Thank you for your response. The graphs that I've created were based on the est. marginal means. Not sure if that answers your question but the scores was ranged from 0-2. What do you mean by predicted values though? $\endgroup$ – vskho2 Oct 17 '13 at 10:50
  • $\begingroup$ The predicted value is the value that the ANOVA predicts for each combination of values. What software are you using? $\endgroup$ – Peter Flom Oct 17 '13 at 11:08
  • 2
    $\begingroup$ To get the predicted value, in SPSS Mixed Model panel, after you have specified all the information, Click Save button, and check the box Predicted values. You should then obtain a new variable after the model is completed. Then you can proceed to see the 32 means, and compare them without treading into the mess of 4- or 5-way interaction terms. $\endgroup$ – Penguin_Knight Oct 17 '13 at 12:22
  • 1
    $\begingroup$ First, try redefining the levels of the within-subject factors, from absolute to relative: same/different as the subject. This should interchange some main effects and interactions. Second, if the only possible values of the dependent variable are 0/1/2 then some of the interactions might be due to floor or ceiling effects, in which case you might try an ordinal logistic analysis. $\endgroup$ – Ray Koopman Oct 17 '13 at 21:44
5
$\begingroup$

Generally, you should start from the highest order interactions. You are probably aware that it is usually not sensible to interpret a main effect A when that effect is also involved in an interaction A:B. This is because the interaction tells you that the effect of A actually depends on the level of B, rendering any simple main effect interpretation of A impossible. In the same way, if you have factors A, B, C, then A:B should not be interpreted if A:B:C is significant.

Thus, when you have a 5-way interaction, none of the lower-order interactions can be sensibly interpreted. Therefore, if I understand you correctly and you have interpreted your lower order interactions, you should probably not continue along those lines.

Rather, what you can do is to split up your data set and continue to analyze factor levels of your data set separately. Which of the factors you use to split up the dataset is arbitrary, but often it is very useful to split up the data for each variable and assess what you see. In your example, you might start with sex, and calculate an ANOVA for males, and another one for females (each ANOVA contains the 4 remaining factors). Just as well, you could split up the data according to ethnicity (one ANOVA for Asian, one for Caucasian). You could also split up by one of the within-subject factors.

I will assume that you have decided to split the data by sex (just to continue with the example here). Then, assume that for males, you get a 4-way interaction. You would then go on to split up the male data by one of the remaining variables (say, ethnicity). You would then calculate ANOVAs for male Asians (over the remaining 3 factors), and for male Caucasians.

Importantly, if you get only a lower-order interaction, then you are only "allowed" to analyze these further. This is because the other factors did not show significant differences. Thus, if your males ANOVA gives you only a 2-way interaction, then you would average over the other factors and calculate only an ANOVA over the 2 interacting factors (and, because we are in the male part of the ANOVAs, this would be for the males alone).

For the females, everything may look different, and so the decision which follow-up ANOVAs to calculate is separate for this group. So, what you did for males should be done for females in the same way ONLY if you got the same interactions.

Thus, you will potentially have a lot of ANOVAs, and it might not be easy to decide which ones to report. You should report 1 complete line down from the hightest interaction to the last effects (possibly t-tests to compare only 1 of your factors at the end). You should not usually report several lines (e.g., one starting the split-up by sex, then another one starting by ethnicity). However, you must report a complete line, and cannot simply choose to report only some of the ANOVAs of that line. So, you report one complete analysis, not more, not less. Which way to go in terms of splitting up / follow-up ANOVA is a subjective decision (unless you have clear hypotheses you can follow), and might depend on which results can be understood best etc.

$\endgroup$
  • $\begingroup$ Thank you for your response thaq. Really appreciate it! In terms of splitting the data (going with the sex example), I have gotten to the stage where I had split the data to male/female Caucasians/Asians. At this stage though, some main effects were significant, some were not, the interaction effects were mostly non-significant and at times both main and interaction effects was not significant as well. However, some the graphs clearly shows that there is a interaction effect but only at one level of the 3rdvariable. For example: Lineup ethnicity x Participant sex at two levels of Lineup Sex $\endgroup$ – vskho2 Oct 17 '13 at 14:10
  • $\begingroup$ Male Caucasians Lineup: Lineup ethnicity x Participant Sex - All main effects & interactions are non sig, Female Caucasian Lineup: Lineup ethnicity x Participant Sex - All main effects & interactions are non.sig But the graph indicates an interaction effect only for Female Caucasians. Would I then mention, "There were no reported sig. effects. However the graph indicates an interaction effect for .. (..then explain with the study theory)"? $\endgroup$ – vskho2 Oct 17 '13 at 14:12
  • $\begingroup$ If I'm understanding your response correctly then, at this stage, if I do not have any significant interaction effects. I would calculate the ANOVA and report it (even when there are sig. main effects). The same applies if everything was not sig. (main effects + interactions)? $\endgroup$ – vskho2 Oct 17 '13 at 14:16
  • $\begingroup$ yes, of course your graph might look as if there might be some effects, but if they do not become significant, then they are not reliable (== they are 'not really there'). Often this becomes understandable if you plot the standard deviation/error around your data points. Large variance will prevent something that looks like an effect/interaction from becoming significant. In such a case, it is incorrect to say that 'there was an interaction in the plot'. Rather, you can say that what looks like an interaction was not significant (and should not be interpreted). $\endgroup$ – thaq Oct 19 '13 at 12:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.