I have conducted an experiment with one group of subjects (n=14). I have two within-subjects factors and one dependent variable (called Error). The factor "Type" has 2 levels (Baseline, Operant); the factor "Event" has two levels (Action, Tone). The hypothesis is that there is a significant interaction Type*Event, specifically because:
- $Error_{(Baseline-Action)}-Error_{(Operant-Action)}\leq 0$
- $Error_{(Baseline-Tone)}-Error_{(Operant_Tone)}\geq0$
So, I first do a two-way repeated measures ANOVA. In R:
aov1 <- aov(Error~Type*Event+Error(subj/(Type*Event)), data=my_data)
summary(aov)
and the interaction is significant. Now I have to do a multiple comparison. I think it is possible to use both a pairwise t test:
# 'inter' is a column created by interaction(my_data$event, my_data$type)
pairwise.t.test(my_data$Error, my_data$inter, paired=T, p.adj="fdr")
and a Tukey test:
model <- lme(value~inter, random=~1|subj/(inter), data=my_data)
comp <- glht(model, linfct=mcp(te="Tukey"),type="none")
summary(comp)
The pairwise confirms my hypothesis 1 and 2, but the Tukey doesn't (it return a significant p-value only for the hypothesis 2). However, I don't know if I'm mistaken something with the commands for the Tukey.
My questions are:
is this modus operandi correct in my case?
on what basis I can choose which among pairwise t test and Tukey's test is the most appropriate? Are there other alternatives?
Thanks.
EDIT. What is the correct procedure for computing effect size from the output of glht (which give z-scores) and from a pairwise comparison t test (which doens't return the t values)?
pairwise.t.test. Type ?pairwise.t.test to see them. Your choice (fdror false discovery rate) applies a different method of multiple comparison control thantukey. There is lots of literature on choosing a method, but, as far as I know, no general consensus on which is best. – Peter Flom Oct 7 '12 at 19:12