# 3-way ANOVA with a significant 2-way interaction. What are my next steps?

I have performed a mixed 3 way ANOVA: Group (2 levels) x Factor2 (2 levels) x Factor3 (2 levels).

There was no significant 3-way interaction; however, there was a significant 2-way interaction between Group and Factor2.

I am confused by the difference between "main effects" and "simple main effects," and I am not sure what step to take next to understand the 2-way interaction.

Do I need to perform a 2-way ANOVA (Group x Factor2)? Do I need to perform 4 separate tests of some kind?

[UPDATE]

Here is the procedure I used to run the simple main effects:

There was no significant 3 way interaction however there was a significant 2 way interaction of Group*Factor1 and a significant main effect of Factor1.

1) Ran through the same menu option as to explore the 3 way and 2 way interactions. SPSS -> Analyze -> GLM -> Repeated Measures.

2) Clicked 'Paste' in the repeated measures window and adjusted the syntax as follows:

GLM AccF1_L1 Acc_F1_L2 Acc_F2_L1 AccF2_L2 BY Group /WSFACTOR=Factor1 2 Polynomial Factor2 2 Polynomial /EMMEANS=TABLES(Factor1*Group) COMPARE(Group).

3) Interpret 'Univariate Tests' 4) If any significant simple main effects go on to interpret pairwise comparisons.

I am going to use R to demonstrate these concepts, but they can be done in any popular statistics software.

## Example Data

The example I like to use is some athletic event. Imagine we have Teams A and B will be facing off against one another. We get people to imagine how happy they will be if Team A wins or if Team B wins. We also ask them if they are a fan of Team A or Team B. Lastly, we get what hand they write with. This gives us four variables: happiness (dependent variable, DV), winning team (first independent variable, IV1), fan of team (IV2), or handedness (IV3).

I simulated data representing this using R:

# generate data ----------------------------------------------------------------
set.seed(1839)
n <- 300
fan <- sample(c("a", "b"), n, TRUE)
win <- sample(c("a", "b"), n, TRUE)
happiness <- rnorm(n, ifelse(fan == win, .5, -.5), 1)
hand <- sample(c("l", "r"), n, TRUE)


The results are programmed to align with what is obvious: The effect of who wins on happiness depends on who the person is a fan of (two-way interaction), but this doesn't depend on handedness (lack of a three-way interaction). People are happy when their team wins, but sad when their team loses.

## Main Effect

The main effect can be seen as the effect of one independent variable on the dependent variable, averaging across all other variables. You could also say that it is the effect ignoring or collapsing across all other variables.

To get the main effect of which team wins (A or B) on happiness, we can do a t-test. This ignores all of the other independent variables.

# main effect ------------------------------------------------------------------
t.test(happiness ~ win)


This returns:

    Welch Two Sample t-test

data:  happiness by win
t = 0.12692, df = 296.16, p-value = 0.8991
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
-0.2346082  0.2669540
sample estimates:
mean in group a mean in group b
-0.08008830     -0.09626121


There is not a significant main effect of which team wins on happiness. This is because it depends on who people are a fan of. We can do that by looking at the interaction.

## Fitting Model

I fit a linear model (the same thing as an ANOVA) to look at the main effects, all two-way interactions, and the three-way interaction.

# fit model --------------------------------------------------------------------
model <- lm(happiness ~ fan * win * hand)
summary(model)


This returns a coefficients table that looks like:

Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept)       0.2060     0.1612   1.278 0.202323
fanb             -0.6378     0.2265  -2.815 0.005203 **
winb             -0.8181     0.2238  -3.655 0.000304 ***
handr             0.1704     0.2280   0.747 0.455450
fanb:winb         1.5237     0.3275   4.652 4.99e-06 ***
fanb:handr       -0.2865     0.3304  -0.867 0.386542
winb:handr       -0.1571     0.3147  -0.499 0.617968
fanb:winb:handr   0.7053     0.4620   1.527 0.127927
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1


The three-way interaction (fanb:winb:handr) is not significant, but one of the two-way interactions (fanb:winb) is. Now we can "probe" this interaction.

## Simple Main Effects

Recall the definition of a main effect:

• The effect of IV1 on the DV, ignoring the effects of all other IVs.

We can change this to the definition of a main effect by changing what comes after the comma:

• The effect of IV1 on the DV, at a specified level of IV2.

These can also be called "conditional" main effects or "conditional average treatment effects" because it looks at the effect of IV1 on the DV given some level of IV2.

So while a main effect says something like, "IV1 does not affect DV," a simple main effect says, "When IV2 = A, then IV1 negatively affects DV." It is that "When IV2 = A" that makes it a simple main effect; it is the main effect of a variable at a given level of another variable (in this case, IV2 = A).

You do not need to fit a different model to examine this. You can estimate it from the model you have. In R, it is as simple as:

# simple main effect -----------------------------------------------------------
library(emmeans)
pairs(emmeans(model, ~ win | fan))


The ~ win | fan is saying that the DV is predicted by (~) the effect of winning, given (|) the level of fandom. This returns:

fan = a:
contrast   estimate        SE  df t.ratio p.value
a - b     0.8966396 0.1573624 292   5.698  <.0001

fan = b:
contrast   estimate        SE  df t.ratio p.value
a - b    -0.9797111 0.1691024 292  -5.794  <.0001


So when fan = a, then Team A winning leads to an increase of happiness over Team B by about .9. On the other hand, when fan = b, the effect of Team A winning leads to a decrease of happiness by about .98. The simple main effect part is in specifying the effect of winning at different levels of fandom.

We can also look at it by handedness if we want, but since there was no significant interaction, then it won't make much of a difference:

> pairs(emmeans(model, ~ win | fan + hand))
fan = a, hand = l:
contrast   estimate        SE  df t.ratio p.value
a - b     0.8180739 0.2237993 292   3.655  0.0003

fan = b, hand = l:
contrast   estimate        SE  df t.ratio p.value
a - b    -0.7056253 0.2391469 292  -2.951  0.0034

fan = a, hand = r:
contrast   estimate        SE  df t.ratio p.value
a - b     0.9752053 0.2212816 292   4.407  <.0001

fan = b, hand = r:
contrast   estimate        SE  df t.ratio p.value
a - b    -1.2537969 0.2391469 292  -5.243  <.0001

• Thankyou for this... I think it's starting to make sense. [Sorry I'm still writing this but saved by accident] – IntoTheBlue Aug 14 '18 at 13:08
• So a simple main effect is conditional, on the level of some IV? I'm still having difficulty proceeding. I am using SPSS. I ran a mixed 3 way ANOVA (GroupF1*F2) each factor having 2 levels. The Within-Subjects effects table shows the three way interaction is non-significant. The same table shows that the GroupF1 interaction is signigicant. How do I drill down into this? According to Laerd I need to investigate Simple Main Effects now, but I'm not sure how I go about this. Sorry for my confusion. This is my first ANOVA. – IntoTheBlue Aug 14 '18 at 15:45
• "So a simple main effect is conditional, on the level of some IV?" Yes, that's it! I haven't used SPSS for a while, but if it's the same as it used to be, then you cannot get these from a drop-down menu. You have to paste the code and then add additional syntax to it. It looks like there's a good amount of tutorials by searching Google for SPSS simple main effects. No apologies necessary—this can be confusing, especially when you're doing your first ANOVA! – Mark White Aug 14 '18 at 15:47
• Ok thankyou! Getting there. Think I'm missing some steps... With this kind of design do I need to work with the Univariate Factorial Anova? Given there is a 2 way interaction does that mean I need to collapse down the within subjects variables? i.e. I currently have variables 1_1, 1_2, 2_1, 2_2. (LevelFactor1_LevelFactor2). If the main effect is significant for Factor 1, and Factor 1 interacts with the Group, would I first average 1_1 and 1_2 together (and 2_1 and 2_2). Does that make sense? i.e. to collapse the factor which isn't in the 2-way interaction? – IntoTheBlue Aug 14 '18 at 17:17
• Yeah, you can use that model with the three-way interaction specified—but when you do simple main effects, you can collapse across the factor which isn't in the 2-way interaction. – Mark White Aug 14 '18 at 17:19