I have a 2x2x2 mixed factorial design with one between-subjects factor (C) and two within-subjects factors (A and B). I analyzed the data with a mixed ANOVA and now want to calculate t-test equivalent for each effect. Please see below for a simulated dataset in R.

If there was no between factor, i.e. the design was fully 2x2 within (factor A and B), I can use the ANOVA contrasts for each effect

  1. main effect factor A (1,1,-1,-1)
  2. main effect factor B (1,-1,1,-1)
  3. interaction AxB (1,-1,-1,1)

and test the resulting difference scores with separate one-sample t-tests against mu=0. This yields equivalent results (same df, t²=F, same p-values) to the full 2x2 within ANOVA.

If a between factor is present, I managed to obtain equivalent t-test results for all ANOVA effects that include the between factor C by performing independent sample t-tests on the DV (main effect of C) and on the difference scores as calculated above (interactions CxA,CxB,CxAxB).

My problem is, that the main effects of the within factors A and B and their interaction AxB, if tested with t-tests against mu=0 as for a full within design, are not anymore equivalent to the effects obtained in the mixed ANOVA.

My question is, why the results form the t-test and the ANOVA slightly diverge and how I can calculate these three t-test in a way that is equivalent to the mixed ANOVA?

Thank you very much for your help!

I used the R-function mixedDesign() by Sven Hohenstein, Reinhold Kliegl, 2009-2015 to simulate the dataset (can be downloaded here http://read.psych.uni-potsdam.de/pmr2/index.php?option=com_content&view=article&id=167:contrast-coding&catid=13:r-playground&Itemid=15)

#Matrix with means for each cell
M.means=matrix(c(0.35,0.55,0.4,0.7,0.35,0.55,0.35,0.55),nrow=2, ncol=4, byrow=T)
#matrix with SDs
M.sd=matrix(c(0.245,0.272,0.317,0.286,0.245,0.272,0.245,0.272),nrow=2, ncol=4, byrow=T)
#list of matrices with correlations of within-subject factors
M1.cor=matrix(c(1.00,0.2,0.5,0,0.2,1.00,0,0.3,0.5,0,1.00,0,0,0.3,0,1.00),nrow=4, ncol=4, byrow=T)
M2.cor=matrix(c(1.00,0.2,0.9,0.2,0.2,1.00,0.2,0.9,0.9,0.2,1.00,0.2,0.2,0.9,0.2,1.00),nrow=4, ncol=4, byrow=T)


#simulate dataset
data=mixedDesign(B=2, W=c(2,2),n=100, M=M.means, SD=M.sd, R=R1,long=T)


   C id  A  B        DV
1 A1  1 a1 b1 0.6617092
2 A1  1 a1 b2 0.9309685
3 A1  1 a2 b1 0.7898644
4 A1  1 a2 b2 0.6968670
5 A1  2 a1 b1 0.8381598
6 A1  2 a1 b2 0.6567378

#mixed ANOVA results

  Effect DFn DFd         F            p p<.05         ges
2      C   1 198  3.692938 5.607905e-02       0.008569437
3      A   1 198 16.299148 7.727887e-05     * 0.008569437
5      B   1 198 99.193180 3.358257e-19     * 0.148958640
4    C:A   1 198 16.299148 7.727887e-05     * 0.008569437
6    C:B   1 198  1.224607 2.698010e-01       0.002156218
7    A:B   1 198  5.205633 2.357750e-02     * 0.002156218
8  C:A:B   1 198  5.205633 2.357750e-02     * 0.002156218

#to get equivalent t-tests:
#set contrasts for effects

#set contrasts for effects
data=data%>%mutate(c_C=ifelse(C=='A1',1,-1),#main effect factor C
                   c_A=ifelse(A=='a2',1,-1),#main effect factor A
                   c_B=ifelse(B=='b2',1,-1),#main effect factor B
                   CxA=c_C*c_A,#interaction CxA
                   CxB=c_C*c_B,#interaction CxB
                   AxB=c_A*c_B,#interaction AxB
                   CxAxB=c_C*c_A*c_B)#interaction CxAxB

#copy DV

#multiply DV with contrasts
#aggregate data (one line per id)

#all ANOVA effects as t-tests
t.test(DV~C,data1,var.equal=T)#matches ANOVA
t.test(data1$c_A,mu=0)#no match
t.test(data1$c_B,mu=0)#no match
t.test(c_A~C,data1,var.equal=T)#matches ANOVA
t.test(c_B~C,data1,var.equal=T)#matches ANOVA
t.test(data1$AxB,mu=0)#no match
t.test(AxB~C,data1,var.equal=T)#matches ANOVA

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.