I am running a mixed factors ANOVA for a brain imaging study on language processing. The design includes four within-subject factors:
- complexity: simple/complex;
- agreement: correct/number violation/gender violation;
- hemisphere: left/right;
- region: anterior/posterior;
and one between-subject factor:
- proficiency: low/high proficiency.
Factors 3 and 4 (hemisphere, region) only matter if they interact with the other factors. This is because the effects of complexity and agreement are predicted to emerge in specific areas of the scalp.
The results reveal a significant 5-way interaction (complexity by agreement by hemisphere by region by group), and I find it almost impossible to figure out what is driving the interaction by just looking at the means.
Am I justified in looking at the two proficiency groups separately? Most similar studies do this, but I'm unclear as to how to do this. For example, once I decide to examine proficiency learners separately, I find a 3-way interaction between agreement, hemisphere, and region in the high-proficiency learners (meaning that there is a brain response for agreement violations, which is captured in the left anterior portion of the scalp), but there is no such interaction for low-proficiency learners (p = .13) Is it licit to report this difference even if the original omnibus ANOVA (with group as a factor) showed that there was no agreement by hemisphere by region by proficiency interaction?
In other words, once I decide to look at the two proficiency groups, can I run the analysis as if I had never compared the two groups or am I constrained by the original ANOVA as to what follow-ups I can do?