t-test with data in long-format including levels of non-interesting factor in R

Question

I've computed an ANOVA with one between-subjects factor (2 groups each including 26 participants) and 2 within-subjects factors (item type with 3 levels and emotion with 2 levels) in ezANOVA. Therefore, my data is ordered in a long-format in which all conditions for each participant are listed underneath each other:

ID     group    itemType    emotion     dependentVariable
p001      A        type1    negative                 3.88
p001      A        type1    neutral                  2.34
p001      A        type2    negative                 5.21
p001      A        type2    neutral                 10.00
...

When I am now computing post-hoc t-tests, e.g. the difference between two item types in a specific group regardless of the emotionality of the items, should I then keep working in this long-format? Or should I change into a wide-format which includes the average over the non-interesting factor, in this case emotionality? I am wondering if in case of the long-format, R might treat the levels of the non-interesting factor as two independent observations and that this computation might then be statistically incorrect?

In case of the long-file, I used this code:

t.test(x = long_file$dependentVariable[long_file$delay=="A" & long_file$itemType=="type1"], 
       y = long_file$dependentVariable[long_file$delay=="A" & long_file$itemType=="type2"], 
       paired = TRUE)

resulting in t=2.43, df=51, p=0.02

I used this code to analyze the dependent variable as average of the non-interesting factor in a wide-format:

t.test(x = wide_file$dependentVariable_type1_averageOverEmotions[wide_file$group=="A"], 
       y = wide_file$dependentVariable_type2_averageOverEmotions[wide_file$group=="A"], 
       paired = TRUE)

resulting in t=2.38, df=25, p=0.03

So, in this case, results don't change a lot regarding the statistical significance, but using the long-format doubles the degrees of freedom.

Answer 1

Since you have a rather complex model with both between factors and within factors, you are better off using a post-hoc analysis that takes the model into account, rather than using a series of t-tests.

I assume you are using R. One approach would be to use the emmeans package, but I don't think ezAnova objects are supported cran.r-project.org/web/packages/emmeans/vignettes/models.html.

You may want to refit your model with a more-supported package, like lme4 or nlme.

Answer 2

My solution for post-hoc tests for ANOVAs computed with ezANOVA: If a variable that contains factor levels in the long format that are not compared in the t-test using t.test in R, then the degrees of freedom are overestimated (doubled in this case). Therefore, in this case, the variable should first be averaged over the non-interesting factor before computing the post-hoc t-test.

For future ANOVAs, I would rather use other functions such as aov which enable post-hoc tests with functions that take the model into account and also correct for multiple comparisons such as emmeans.