2
$\begingroup$

I have conducted a 2-way ANOVA and obtained a significant Main Effect in IV1 and IV2, but a non-significant interaction. Following that, I have analyzed IV1 using a test for Simple Effects and received a significant Simple effect at level 1 and a non-significant simple effect at level 2.

To speak in particularities of my experiment: I tested the main effect of sex (factor 1) and treatment (factor 2). The interaction between the main effects was non-significant (p = .279), but the main effects were both individually significant (sex, p = .008 and treatment, p = .022). I proceeded to run a simple effects test and identified significance in males (p = .016) and non-significance in females (p = .388). Without the simple effects test I wouldn't be able to distinguish the efficacy of drug treatment in males versus females. Surely the simple effects test was necessary to run in this case, contrary to the non-significant main interaction? Is there something I am missing that would counter my interpretation of the treatment being non-efficacious in females?

I am under the understanding that because of the non-significant interaction, completing a simple effect test was deemed "unnecessary". I don't understand why the simple effect test is only completed following the presence of a significant interaction of main effects. Can anybody explain how the Simple Effect test is influenced by the interaction, and how I would interpret my Simple Effect results accordingly?

Improve this question
$\endgroup$
11
  • $\begingroup$ Are you sure you are using the terms "significant" and "non-significant" in the way you intend? Your understanding would be consistent with interpreting these terms in the opposite of their usual meanings. $\endgroup$
    – whuber ♦
    Jan 14, 2021 at 15:50
  • $\begingroup$ No, I reported the significance of each Effect and Interaction accurately. Are you suggesting that my second paragraph is incorrect? i.e. Simple Effect IS supposed to be tested when the interaction of IV1 and IV2 is non-significant? $\endgroup$
    – Hecealdor
    Jan 14, 2021 at 15:58
  • 2
    $\begingroup$ @whuber: You seem to interpret simple effect as main effect. I recently learned this is used in another way, see glimo.vub.ac.be/downloads/simpleeffect.htm, as a main effect but restricting to only some level of some other variable. Simple slope seems to be used in the same sense. $\endgroup$
    – kjetil b halvorsen ♦
    Jan 15, 2021 at 13:21
  • 1
    $\begingroup$ @kjetil Thank you: that looks right. (I had begun to wonder whether there was some difference of interpretation here, which is why I tried to make mine clear in my previous comment.) Now the question makes sense, if we take the colleague's advice to mean "because you will proceed as if there is no interaction, there is no point in distinguishing simple effects from main effects." $\endgroup$
    – whuber ♦
    Jan 15, 2021 at 14:10
  • 1
    $\begingroup$ Think about this: When you split the sample in subgroups and do multiple tests, you get lower power. Please also show us your confidence intervals! (and sample sizes.) This Gelman blog post is relevant. Read it! $\endgroup$
    – kjetil b halvorsen ♦
    Jan 16, 2021 at 15:53

1 Answer 1

Reset to default
1
$\begingroup$

Simple effects are defined as the effect of one variable, but restricted only to some level of some other variable. This meaning is used in for instance the R package (on CRAN) phia and discussed in its vignette. Another term which seems to be used with the same meaning is Simple slope.

For a simple illustration, let there be two factors, a and b with interaction a:b. If they have each two levels in a factorial structure, there are two simple effects for a (and likewise for b), one for each level of b. The interaction is the difference between the two simple effects. So, if the interaction is zero, the simple effects are equal (and equal to the main effect.) It is in this sense that there is no need to study simple effects when there is no interaction.

Much the same is true when the factors have more than two levels (or with numerical variables), but the details are more involved. See for instance the vignette linked above.

Improve this answer
$\endgroup$
2
  • $\begingroup$ I'll speak in context of my experiment so it makes more sense.I tested the main effect of sex (factor 1) and treatment (factor 2). The interaction between the main effects was non-significant, but the main effects were both individually significant. I proceeded to run a simple effects test and identified significance in males and non-significance in females. Without the simple effects test I wouldn't be able to distinguish the efficacy of drug treatment in males versus females. Surely the simple effects test was necessary to run in this case, contrary to the non-significant main interaction? $\endgroup$
    – Hecealdor
    Jan 15, 2021 at 21:48
  • 1
    $\begingroup$ Can you please add this new information as an edit to the Q? Not everybody reads comments. And, some numerical details could be interesting, for a more detailed response. I will try to edit my answer tomorrow, now its night here ... $\endgroup$
    – kjetil b halvorsen ♦
    Jan 16, 2021 at 3:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.