1
$\begingroup$

I conducted a 2 x 4 within-subjects design experiment. I presented participants with unpleasant and neutral pictures and asked them to cope with their emotions in four different ways. Therefore, the first independent variable refers to picture valence (unpleasant vs. neutral pictures) and has two levels while the other one refers to emotion regulation (ER) and has four levels (1, 2, 3, 4, where 1 is a control condition). I measured various things which resulted in a few dependent variables. My a priori hypotheses where that for every dependent variable I will find differences between a control condition and any other condition (1 vs. 2; 1 vs. 3; 1 vs. 4) only for unpleasant images. That is why I decided to run planned contrasts for these comparisons.The overall ANOVA is not significant (p = .081) but one of the contrasts is (p = .049). I know that I can find significant differences when comparing two conditions at a time even if the overall ANOVA is not significant. However, there seems to be no agreement on whether it is ok to look at the contrasts when ANOVA is not significant. What is more, there seems to be a disagreement on whether a correction for multiple comparisons should be applied when running planned contrasts. My question is if I should say that I found a significant result between the two conditions? I have doubts about it because the overall ANOVA is not significant and the difference between the two conditions is found only when no correction for multiple comparisons is applied. On the other hand, I am not running any other comparisons and I predicted this difference in advance. Thanks

$\endgroup$
  • $\begingroup$ If ANOVA is nsig the (rare occasional) sig pairwise subtest should not be honoured. (Else why do ANOVA at all?: if you do it, it means you respect its null hypothesis worth to be tested and respect its assumptions). $\endgroup$ – ttnphns Oct 5 '16 at 10:55
  • $\begingroup$ Hi, thank you for the answer. To be honest I looked at the overall Anova just because it pops up when you run contrasts. I didn't actually mean to interpret it if interpreting contrasts exclusively was possible. $\endgroup$ – Karolina Oct 5 '16 at 11:57
1
$\begingroup$

If you pre-specify something then that is what you should do. Otherwise why pre-specify it? In this case if you have a strong theoretical justification for looking at just those contrasts then do it. Ideally this would all have been recorded somewhere in advance like your ethical approval document or your published protocol so nobody can accuse you have being post-hoc.

$\endgroup$
  • $\begingroup$ Thank yo for the response. But in such a case would you say the result (p = .049) is significant or would you rather correct for multiple comparisons (.05/3) and say than it's not? $\endgroup$ – Karolina Oct 5 '16 at 17:19
  • $\begingroup$ The whole issue of adjustment for multiplicity is much discussed. Go to the tag multiple-comparisons on this site and ask for the highest voted and read some of them. You will find much heat and little light. I personally would not normally adjust but other people find that a heresy. Sorry not to be able to give you a yes/no answer. $\endgroup$ – mdewey Oct 5 '16 at 17:32
0
$\begingroup$

If that would be my research question and I knew all my hypothesis most likely I'd look at 95% CI (for both means and contrast coef.) and see how they plot all together (looking for overlaps and band widths).

$\endgroup$
  • $\begingroup$ Thank you for the answer. Do you mean you'd just examine it visually? $\endgroup$ – Karolina Oct 5 '16 at 11:58
  • $\begingroup$ Yes,have a look at this text if you are not certain on how to read overlaps: ]psychologicalscience.org/index.php/publications/observer/2010/… $\endgroup$ – blazej Oct 5 '16 at 12:53
  • $\begingroup$ Thanks for the text. But I think should't look at overlaps in within-subjects design, should I? $\endgroup$ – Karolina Oct 6 '16 at 14:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.