5
$\begingroup$

I am conducting a 2x2 within-subjects design. When I plot my results it appears that there will be an interaction between my variables but unfortunately none is emerging p=0.08. I find it a shame that I can't explore this further with simple main effects. Does anyone have any suggestions?

$\endgroup$
3
  • 5
    $\begingroup$ We could probably use some additional context here - what are you studying? What's your sample size? Is there a theoretical reason to believe this interaction exists? Is this exploratory or confirmatory analysis? Because I can think of a great many contexts in which I would say that p = 0.08 is plenty of evidence that that interaction exists. $\endgroup$ Sep 13 '11 at 19:31
  • 1
    $\begingroup$ It's also not clear exactly what your question is. Are you looking for help in exploring the interaction, or just justification for doing so? $\endgroup$ Sep 13 '11 at 20:20
  • 1
    $\begingroup$ Who says you can't? $\endgroup$
    – rolando2
    Sep 13 '11 at 21:37
9
$\begingroup$

Worship not p = 0.05. Explore away.

Additionally, in some contexts, relying on p = 0.05 for an interaction threshold is actually a bit flawed, as interaction tests are typically fairly low powered, and you can and should be using a somewhat higher threshold to accept statistical evidence of interaction. Sander Greenland or Miguel Hernan undoubtedly have a paper discussing the problem.

$\endgroup$
2
  • $\begingroup$ anti-conservative? What you said seems to indicate the $p=.05$ is conservative. $\endgroup$
    – Macro
    Sep 13 '11 at 21:04
  • $\begingroup$ Edited slightly for clairity - anti-conversative slipped in there because its currently being used in a paper open in another window. sheepish $\endgroup$
    – Fomite
    Sep 13 '11 at 22:41
5
$\begingroup$

"Can't"? Who says you can't? There's nothing magic about p = .05. You can certainly explore the interaction.

The question is how you deal with complaints from people who say you can't do this. In addition to works by Greenland or Hernan (see @EpiGrad's response) you can look for papers by Jacob Cohen or Paul Meehl or the book "The Cult of Statistical Significance" by Ziliak or the book "Statistics as Principled Argument" by Abelson

$\endgroup$
2
$\begingroup$

You might consider switching to mixed effects modelling, which in some cases provides superior power over ANOVA.

$\endgroup$
2
$\begingroup$

I agree with the others that you certainly can explore this interaction, but, if it's not significant, you might not have much power to aid your analysis.

$\endgroup$
2
$\begingroup$

To say something similar to the other answers in slightly different words.

I would do the following:
Report that the (hopefully expected) interaction is almost or marginal significant or that there is a trend towards significance (these expressions are all common, at least in psychology). Then, state that therefore I inspect this interaction further with follow up simple main effects analyses or contrasts.

It is absolutely no problem to do so, if your main hypotheses are within in this interaction. As said before, omnibus tests of interaction do not have the highest power.

See also here.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.