0
$\begingroup$

Stats textbooks almost always tell us a significant interaction is necessary if we want to test the simple main effects[1]. But why? Is this again a way to control the error rate? If so, significant interactions are not indeed a must-have-first, since we can control error rates using, say, Bonferroni correction.

[1] See http://glimo.vub.ac.be/downloads/simpleeffect.htm for an example.

$\endgroup$
  • $\begingroup$ Please provide an explicit source for your claim, with a quote and full reference or link. I don't recall ever seeing such a claim being made (I've seen claims with some of the same words, but saying something different to this). $\endgroup$ – Glen_b Sep 14 '17 at 2:09
  • 1
    $\begingroup$ Thanks for providing the link, but I don't actually see the claim you're talking about made there. The author discusses simple effects testing as a typical follow-up to a significant interaction, but never suggests that the interaction is a "must-have-first". Am I missing something? $\endgroup$ – Rose Hartman Sep 14 '17 at 8:39
  • $\begingroup$ Is it possible that you have read something along the lines "if there is only one categorical variable, you may do just t-tests, but when there are two categoricals and perhaps an interation, you need an ANOVA"? $\endgroup$ – David Ernst Sep 14 '17 at 22:08
2
$\begingroup$

Note: my answer here relates to ordinary main effects rather than to simple effects, which turn out to be a terminology for the effect of one factor at a single level of the other factor.

Statistics textbooks actually say the opposite of this. In a factorial design with two or more factors, it isn't sensible to test for main effects in presence of an interaction because the main effects have no interpretable meaning in that case. In other words, you need to make sure that the interaction is nonsignificant before you go ahead and test for simple main effects.

This rule is called the Principle of Marginality and was first articulated as a general principle by John Nelder in his 1977 paper on linear models in the Journal of the Royal Statistical Society. In very simple terms, an interaction means that the effect of each factor depends on the other so, when interaction is present, there is no way to summarize the effect of any of the individual factors in isolation (which is what a simple main effect is trying to do).

Another way to think about the Principle of Marginality is that a statistical model should always include all the main effects and lower order interactions whenever a higher order interaction is included. In other words, the main effects and lower order interactions are part of the total package implied by any high order interaction.

So one needs to think hierarchically about any statistical model. Do you need all the complexity of an interaction model (with both main effects and interaction)? If yes, then no simplification is possible and you have to interpret the interactions. If no, then you can remove the interaction and start testing for the main effects. What you can't do is to make a sensible model with an interaction but missing one of the corresponding main effects --- that would contravene the marginality principle.

I don't want to be trite, but it's not misleading to think of an interaction as like a physical arch that sits on two posts. You can remove the arch while keeping the posts, but you can't remove either of the posts without removing the arch as well.

Reference

Nelder, J. A. (1977). A reformulation of linear models. Journal of the Royal Statistical Society 140 (1): 48–77.

Nelder, J. A. (2000). Functional marginality and response-surface fitting. Journal of Applied Statistics 27 (1): 109-112. http://dx.doi.org/10.1080/02664760021862

$\endgroup$
  • $\begingroup$ "Simple (main) effect" here doesn't mean "main effect". Simple effect is " the effect of one independent variable within one level of a second independent variable." (web.pdx.edu/~newsomj/da1/ho_factorial%20follow-ups.pdf) So I'm not contradicting Principle of Marginality. $\endgroup$ – Shelling Sep 14 '17 at 7:30
  • $\begingroup$ @Shelling OK, I see. "Simple main effect" is not traditional terminology in statistics, so it will be misunderstood by many readers, especially by non SPSS users or by older statisticians. I for one have computed simple effects for decades but never called them such. I suggest you expand your question to explain explicitly what a simple main effect is and to give your question more context. Also note Glen_b's request. $\endgroup$ – Gordon Smyth Sep 14 '17 at 8:10
  • $\begingroup$ @Shelling If you expand and clarify your question, then I will delete my answer. $\endgroup$ – Gordon Smyth Sep 14 '17 at 8:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.