1
$\begingroup$

I conducted an experiment to investigate the effects of musical complexity on visual attention and am now analyzing my data.

I ran a two-way repeated measure ANOVA, with my two factors being Musical condition (control, simple, complex), and the second factor being flanker type (neutral, congruent, incongruent). Now the results indicate a significant difference for both factors but also their interaction (my DV is reaction time). Pairwise comparisons tell me which musical conditions are significantly different from each other, and the same goes for flanker type.

But for the interaction between them, Musical Condition*Flanker Type (such that I want to know whether there were significant difference between Simple Music+neutral flankers vs. Complex Music+neutral flankers for example), am I correct in saying that I have to run a paired t-tests? If I want to compare each condition combination to each other, I would have to run 9 paired t-tests, correct?

If I do that, would I have to divide my significance level of 0.05/9 to account for family wise error?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

I would first run simple omnibus tests of the effect of one variable conditional on the levels of the other. That would be analogous to running three one-way ANOVAs where you test the effect of Musical condition on response time for the neutral level of flanker, for the congruent level of flanker and for the incongruent level of flanker. The only difference is that if homogeneity of variance is assumed, you would use MS_error (aka MS_within) from the two-way design and its degrees of freedom as the denominator for testing each one-way ANOVA.

In general, if we give the conventional 0.05 to each main effect and the interaction, then each simple effect should get 2*0.05 = 0.10 because, for example, $SS_{B|A} = SS_{B} + SS_{A \times B}$. That is, the sum of squares for the simple effect is equal to the sum of the sum of squares for the main effect plus the interaction.

Thus, each of your three simple omnibus tests would use an alpha significance level of 0.10/3 = 0.033. Any significant simple one-way ANOVAs should be followed up with pairwise comparisons. The non-significant simple one-way ANOVAs should not be followed up on.

For the follow-up pairwise comparison, I would use J.P. Shaffer's logical procedure for planned post-omnibus pairwise comparisons. In the three group case, this is equivalent to Fisher's LSD (i.e., no adjustment; see Levin, Serlin, & Seaman, 1994, Psych. Bulletin for a citation).

Improve this answer
$\endgroup$
6
  • $\begingroup$ Should I compare each of the three music conditions to one flanker type per ANOVA, or the other way around (compare one music condition per ANOVA to three levels of the flanker type)? $\endgroup$
    – The_Dude
    Commented Dec 26, 2014 at 21:55
  • 1
    $\begingroup$ With this approach, you won't be comparing the music conditions to one flanker type, you will be comparing the music conditions to themselves, conditional on the level of flanker type. The choice to make is about which factor to condition on. Go with the one that makes the most sense in the context of your research question and is the most straightforward for interpretation. $\endgroup$
    – bsbk
    Commented Dec 26, 2014 at 22:05
  • $\begingroup$ For the sake of my research report, do you have any citations for how you arrived at the "then each simple effect should get 2*0.05 = 0.10 because, for example, SSB|A=SSB+SSA×B" ? $\endgroup$
    – The_Dude
    Commented Dec 26, 2014 at 22:18
  • $\begingroup$ I don't have the book with me but I believe Keppel and Wickens, 2004, discuss this approach in their chapter on simple effects. $\endgroup$
    – bsbk
    Commented Dec 26, 2014 at 22:34
  • $\begingroup$ Two of my three simple one-way ANOVAS were significant at the adjusted alpha 0.033. Should the follow up pairwise-comparisons continue to use the family-wise adjusted alpha (0.033) or revert back to using 0.05? $\endgroup$
    – The_Dude
    Commented Jan 30, 2015 at 12:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.