# t-test with data in long-format including levels of non-interesting factor in R

I've computed an ANOVA with one between-subjects factor (2 groups each including 26 participants) and 2 within-subjects factors (item type with 3 levels and emotion with 2 levels) in ezANOVA. Therefore, my data is ordered in a long-format in which all conditions for each participant are listed underneath each other:

ID     group    itemType    emotion     dependentVariable
p001      A        type1    negative                 3.88
p001      A        type1    neutral                  2.34
p001      A        type2    negative                 5.21
p001      A        type2    neutral                 10.00
...


When I am now computing post-hoc t-tests, e.g. the difference between two item types in a specific group regardless of the emotionality of the items, should I then keep working in this long-format? Or should I change into a wide-format which includes the average over the non-interesting factor, in this case emotionality? I am wondering if in case of the long-format, R might treat the levels of the non-interesting factor as two independent observations and that this computation might then be statistically incorrect?

In case of the long-file, I used this code:

t.test(x = long_file$$dependentVariable[long_file$$delay=="A" & long_file$$itemType=="type1"], y = long_file$$dependentVariable[long_file$$delay=="A" & long_file$$itemType=="type2"],
paired = TRUE)


resulting in t=2.43, df=51, p=0.02

I used this code to analyze the dependent variable as average of the non-interesting factor in a wide-format:

t.test(x = wide_file$$dependentVariable_type1_averageOverEmotions[wide_file$$group=="A"],
y = wide_file$$dependentVariable_type2_averageOverEmotions[wide_file$$group=="A"],
paired = TRUE)


resulting in t=2.38, df=25, p=0.03

So, in this case, results don't change a lot regarding the statistical significance, but using the long-format doubles the degrees of freedom.

Since you have a rather complex model with both between factors and within factors, you are better off using a post-hoc analysis that takes the model into account, rather than using a series of t-tests.

I assume you are using R. One approach would be to use the emmeans package, but I don't think ezAnova objects are supported cran.r-project.org/web/packages/emmeans/vignettes/models.html.

You may want to refit your model with a more-supported package, like lme4 or nlme.

• Thanks for the response. I ran other post-hoc analyses and those t-tests are just one part of it. I did try linear mixed models, but I decided to use ANOVAs as I only have categorical factors, one with 3(!) levels, and the ANOVA-results are therefore much easier to interpret. I already found my answer and will always use the mean of an uninteresting factor in case of t-tests. Thanks anyways! Feb 17, 2022 at 16:26
• Anova is just a form of general linear model. You should be able to get an anova-like output for models fit with common packages. ... In any case, it's worth your while to learn to use packages like emmeans or multcomp for post-hoc comparisons. They represent flexible and modern approaches, and reflect the fitted model under consideration. Feb 17, 2022 at 16:58
• I will have a look into emmeans and multcomp, thanks for the suggestion! Feb 18, 2022 at 12:12

My solution for post-hoc tests for ANOVAs computed with ezANOVA: If a variable that contains factor levels in the long format that are not compared in the t-test using t.test in R, then the degrees of freedom are overestimated (doubled in this case). Therefore, in this case, the variable should first be averaged over the non-interesting factor before computing the post-hoc t-test.

For future ANOVAs, I would rather use other functions such as aov which enable post-hoc tests with functions that take the model into account and also correct for multiple comparisons such as emmeans.

• What is seriously missing here is any conception of what is the underlying statistical model. It is really important to understand that, otherwise one is following a blind path. Feb 19, 2022 at 16:29