Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I have a run an unbalanced 2x2x2x2 Type II ANOVA in R and am having trouble following up on the results. Here is the output:

       Effect DFn DFd             F           p p<.05             ges
2           cond   1 127  3.2359349424 0.074414031       0.0110769653158
3             sf   1 127  1.6981345415 0.194889782       0.0058436648717
5             ba   1 127  1.5404865586 0.216833055       0.0012264293759
9             tt   1 127  1.9253260611 0.167700584       0.0054448755666
4        cond:sf   1 127  0.1846599042 0.668127012       0.0006387833954
6        cond:ba   1 127  5.8799698820 0.016721105     * 0.0046651103251
7          sf:ba   1 127  1.4992638464 0.223051114       0.0011936498636
10       cond:tt   1 127  0.5266890712 0.469337439       0.0014954062256
11         sf:tt   1 127  0.0768302867 0.782090431       0.0002184199961
13         ba:tt   1 127 11.5004885802 0.000927851     * 0.0087996011237
8     cond:sf:ba   1 127  0.0042138896 0.948344162       0.0000033589171
12    cond:sf:tt   1 127  0.1197411309 0.729888009       0.0003403692520
14    cond:ba:tt   1 127  0.3878677814 0.534539033       0.0002993221940
15      sf:ba:tt   1 127  0.0001339682 0.990783282       0.0000001034158
16 cond:sf:ba:tt   1 127  0.4820119706 0.488780454       0.0003719473651

cond and sf are between-subjects factors. ba and tt are within (or repeated). The unbalanced nature of the experiment has meant that I have used a type II anova.

You can see that we have a suggestively significant main effect of cond (2) and two significant interactions (6 & 13). I have graphed the interactions and they seem logical, but I am sure that cond is contributing somewhat to the interactions.

I am at a loss at how to proceed. I suppose I wish to do some kind of post-hoc analysis concentrating on the interactions. I have investigated a number of different R packages (afex, phia, contrasts etc), but have yet to work out what I am actually doing with these interactions.

My data looks like this:

str(xx)
'data.frame':   524 obs. of  6 variables:
$ ba  : Factor w/ 2 levels "before","after": 1 1 1 1 1 1 1 1 1 1 ...
    $ tt  : Factor w/ 2 levels "targ","calm": 1 1 1 1 1 1 1 1 1 1 ...
$ p   : Factor w/ 131 levels "1","2","3","4",..: 4 8 9 10 13 18 19 22 25 29 ...
    $ cond: Factor w/ 2 levels "Control","Spider": 1 1 1 1 1 1 1 1 1 1 ...
$ sf  : Factor w/ 2 levels "Fear","No-Fear": 1 1 1 1 1 1 1 1 1 1 ...
    $ eda : num  1.478 -0.56 -0.27 -0.902 -0.483 ...

Moreover consider Thompson (2006) :

As noted by Rosnow and Rosenthal (1989a), the cell means “are the combined effects of the interaction, the row effects [a main effect], the column effects [a second main effect], and the grand mean” (p. 144). By the same token, simple post hoc tests of the cell means also do not yield insight about the origins of interaction effects, because the interaction effects are not uniquely a function of the cell means (Boik, 1979).

So I guess post-hoc t-tests (with adjusted p-values) are out?

Update: Following advice I have found from @henrik (here) I have been investigating the phia package and the testInteractions function. I have been getting some results (for a type III anova - so not the type II I am after) but, again, I am way out of my depth here:

e.g.,

> testInteractions(m2[["lm"]], pairwise = "ba", "cond", idata = m2[["idata"]],     adjustment = "none")              
Multivariate Test: Pillai test statistic
P-value adjustment method: none
                      Value Df test stat approx F num Df den Df  Pr(>F)  
after-before : Control -0.31910  1  0.045305   6.0268      1    127 0.01544 *
after-before :  Spider  0.14243  1  0.008045   1.0300      1    127 0.31208  
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 

Latest Update:

I seem to be getting further bogged down with this. Using lmer as referenced here gives this:

fit.1 <-lmer(eda ~ 1 + cond * sf * ba *tt +(ba *tt | p), xx)

which leads to:

> anova(fit.1)
Analysis of Variance Table
          Df  Sum Sq Mean Sq F value
cond           1 0.00330 0.00330  0.0158
sf             1 0.04236 0.04236  0.2035
ba             1 0.07408 0.07408  0.3559
tt             1 0.34651 0.34651  1.6649
cond:sf        1 0.05633 0.05633  0.2707 
cond:ba        1 2.38017 2.38017 11.4359
sf:ba          1 0.45349 0.45349  2.1789
cond:tt        1 0.03832 0.03832  0.1841
sf:tt          1 0.03197 0.03197  0.1536
ba:tt          1 2.39403 2.39403 11.5025
cond:sf:ba     1 0.02595 0.02595  0.1247 
cond:sf:tt     1 0.00667 0.00667  0.0320
cond:ba:tt     1 0.08078 0.08078  0.3881
sf:ba:tt       1 0.00003 0.00003  0.0001
cond:sf:ba:tt  1 0.10034 0.10034  0.4821

Which seems to be confirming the earlier finding of a significant interaction between cond * ba and ba * tt (but is it type II?). Still not sure if any of this is correct.

To clarify then: I am looking for advice about what to do next, in terms of understanding the significant interactions.

share|improve this question
    
In response to your flag, no you can't delete your question while the bounty is running. But (1) you can post your solution so that users will learn from it, and (2) you can wait for possible alternative solution, if any, so that you don't necessarily have to award the bounty to yourself (that's really up to you). –  chl Jan 16 '13 at 22:24
    
Thanks for clarifying. –  Frank_Zafka Jan 16 '13 at 22:38
add comment

1 Answer

up vote 4 down vote accepted
+100

As this is still a very simple design you should stick with the classical analysis, ANOVA.

I recommend (in agreement with e.g., Maxwell & Delaney) to use Type III sums of squares for this problem and inspect the interactions by running separate ANOVAs for the levels for one of the factors involved in the interactions (similar to so called simple main effects analysis). This will essentially tell you what drives the interaction and it is convenient to report (it is generally recommended to plot the interaction before doing so).

Note that you need to use contrast coding before running ANOVAs in R with the 3 sums of squares by calling the following before your ANOVAs:

options(contrasts=c("contr.sum","contr.poly"))
share|improve this answer
    
All sounds very good. ;) –  Frank_Zafka Jan 17 '13 at 8:22
1  
(+1) I believe contr.sum is called deviation coding. –  chl Jan 17 '13 at 10:39
    
@chl I think there is no fully agreed upon naming convention. In Judd, Westfall, and Kenny (2012) it is called "contrast or effect" coding (footnote 7). Aiken & West (1991) call it effects coding only. If I find my Maxwell & Delaney, I will look it up there. –  Henrik Jan 17 '13 at 10:52
    
I assume that running ezANOVA with type=3 does the same thing? –  Frank_Zafka Jan 17 '13 at 10:58
1  
No. Only if you set the correct contrasts beforehand by running the command: options(contrasts=c("contr.sum","contr.poly")) –  Henrik Jan 17 '13 at 13:29
show 5 more comments

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.