0
$\begingroup$

I have a 2x2x2 mixed factorial design with one between-subjects factor (C) and two within-subjects factors (A and B). I analyzed the data with a mixed ANOVA and now want to calculate t-test equivalent for each effect. Please see below for a simulated dataset in R.

If there was no between factor, i.e. the design was fully 2x2 within (factor A and B), I can use the ANOVA contrasts for each effect

  1. main effect factor A (1,1,-1,-1)
  2. main effect factor B (1,-1,1,-1)
  3. interaction AxB (1,-1,-1,1)

and test the resulting difference scores with separate one-sample t-tests against mu=0. This yields equivalent results (same df, t²=F, same p-values) to the full 2x2 within ANOVA.

If a between factor is present, I managed to obtain equivalent t-test results for all ANOVA effects that include the between factor C by performing independent sample t-tests on the DV (main effect of C) and on the difference scores as calculated above (interactions CxA,CxB,CxAxB).

My problem is, that the main effects of the within factors A and B and their interaction AxB, if tested with t-tests against mu=0 as for a full within design, are not anymore equivalent to the effects obtained in the mixed ANOVA.

My question is, why the results form the t-test and the ANOVA slightly diverge and how I can calculate these three t-test in a way that is equivalent to the mixed ANOVA?

Thank you very much for your help!

I used the R-function mixedDesign() by Sven Hohenstein, Reinhold Kliegl, 2009-2015 to simulate the dataset (can be downloaded here http://read.psych.uni-potsdam.de/pmr2/index.php?option=com_content&view=article&id=167:contrast-coding&catid=13:r-playground&Itemid=15)

#Matrix with means for each cell
M.means=matrix(c(0.35,0.55,0.4,0.7,0.35,0.55,0.35,0.55),nrow=2, ncol=4, byrow=T)
#matrix with SDs
M.sd=matrix(c(0.245,0.272,0.317,0.286,0.245,0.272,0.245,0.272),nrow=2, ncol=4, byrow=T)
#list of matrices with correlations of within-subject factors
M1.cor=matrix(c(1.00,0.2,0.5,0,0.2,1.00,0,0.3,0.5,0,1.00,0,0,0.3,0,1.00),nrow=4, ncol=4, byrow=T)
M2.cor=matrix(c(1.00,0.2,0.9,0.2,0.2,1.00,0.2,0.9,0.9,0.2,1.00,0.2,0.2,0.9,0.2,1.00),nrow=4, ncol=4, byrow=T)

R1=list(M1.cor,M2.cor)

#simulate dataset
set.seed(1)
data=mixedDesign(B=2, W=c(2,2),n=100, M=M.means, SD=M.sd, R=R1,long=T)

names(data)=c('C','id','A','B','DV')
head(data)

   C id  A  B        DV
1 A1  1 a1 b1 0.6617092
2 A1  1 a1 b2 0.9309685
3 A1  1 a2 b1 0.7898644
4 A1  1 a2 b2 0.6968670
5 A1  2 a1 b1 0.8381598
6 A1  2 a1 b2 0.6567378


#mixed ANOVA results
library(ez)
ezANOVA(data,DV,wid=id,within=.(A,B),between=C,type=3)


$`ANOVA`
  Effect DFn DFd         F            p p<.05         ges
2      C   1 198  3.692938 5.607905e-02       0.008569437
3      A   1 198 16.299148 7.727887e-05     * 0.008569437
5      B   1 198 99.193180 3.358257e-19     * 0.148958640
4    C:A   1 198 16.299148 7.727887e-05     * 0.008569437
6    C:B   1 198  1.224607 2.698010e-01       0.002156218
7    A:B   1 198  5.205633 2.357750e-02     * 0.002156218
8  C:A:B   1 198  5.205633 2.357750e-02     * 0.002156218


#to get equivalent t-tests:
#set contrasts for effects
library(dplyr)

#set contrasts for effects
data=data%>%mutate(c_C=ifelse(C=='A1',1,-1),#main effect factor C
                   c_A=ifelse(A=='a2',1,-1),#main effect factor A
                   c_B=ifelse(B=='b2',1,-1),#main effect factor B
                   CxA=c_C*c_A,#interaction CxA
                   CxB=c_C*c_B,#interaction CxB
                   AxB=c_A*c_B,#interaction AxB
                   CxAxB=c_C*c_A*c_B)#interaction CxAxB

#copy DV
data$DV_copy=data$DV

#multiply DV with contrasts
data1=data%>%ungroup()%>%mutate_at(c('c_C','c_A','c_B','CxA','CxB','AxB','CxAxB'),funs(.*DV))
#aggregate data (one line per id)
data1=data1%>%group_by(id,C)%>%summarise_at(c('DV','c_C','c_A','c_B','CxA','CxB','AxB','CxAxB'),sum)

#all ANOVA effects as t-tests
t.test(DV~C,data1,var.equal=T)#matches ANOVA
t.test(data1$c_A,mu=0)#no match
t.test(data1$c_B,mu=0)#no match
t.test(c_A~C,data1,var.equal=T)#matches ANOVA
t.test(c_B~C,data1,var.equal=T)#matches ANOVA
t.test(data1$AxB,mu=0)#no match
t.test(AxB~C,data1,var.equal=T)#matches ANOVA
$\endgroup$

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.