4
$\begingroup$

I am analyzing two within-subject categorical variables (Factor A and Factor B) in R. Using linear mixed effects, I got a significant interaction. When I start to analyze the simple effect, I firstly used t.test, and then used the emmeans package. However, I got different results. I don't know which one I should trust. Particularly, I want to compare B1 and B2 on the level of A1. The following is the code I have in R:

emm1 = emmeans(model, ~ A * B)
emm1
pairs(emm1, simple = "B")

Results:

structure = A1:
   contrast estimate    SE  df t.ratio p.value
   B1 - B2     0.451 0.140 395   3.230  0.0013

For the t.test, I firstly subset the data, and run the t.test:

datasubset = data[data$A == "A1", ]
datasub.t=t.test(dv~ B, data= datasubset)

Results:

t = 1.8013, df = 188.59, p-value = 0.07325

So I got different results, and which one should I trust? Or which step that might be incorrect in the code leads to the different results?

$\endgroup$

1 Answer 1

8
$\begingroup$

You said your conditions are within-subject but you did an independent samples t-test. If you do a paired t-test (i.e., setting paired = TRUE in the call to t.test()), the results will be closer, but still not the same. This is because your repeated measures ANOVA (what I assume you did, but you didn't show the code for it) uses the residual sums of squares across all conditions, whereas the t-test only uses the data from the slice you selected. You should use emmeans and not the t-test if you want accurate results.

EDIT given comments: Because your model has two random effects, a t-test, paired or otherwise, is not appropriate to test your slice hypothesis. Again, emmeans was specifically designed to test these hypotheses, so use it.

$\endgroup$
4
  • 1
    $\begingroup$ Thanks so much! I forgot the command paired=T. The model I used is linear mixed effect, and the code is like the following:data=lmerTest::lmer (dv~factorA*factorB + (1|subject)+(1|item), data = data_, REML=FALSE) I am not sure which one I should use if I want to test the simple effect, the t.test or the emmeans command? What I did is to subset the data and use the t.test. And the alternative that I did is to use the emmeans based on the linear mixed effect mode. Thank you so much for your answer in advance. $\endgroup$
    – Buffoon
    Apr 24, 2023 at 2:13
  • $\begingroup$ Hi Noah, by the way, I have another question concerning the format of the data. After subsetting the data, the data is still long format. Is it the appropriate format to perform a paired-t in R? Because usually in SPSS I split the two levels of a variable into two columns when performing a paired-t test. I am new in this area, and there are many things that I am not sure of. Your help is very much appreciated. $\endgroup$
    – Buffoon
    Apr 24, 2023 at 4:06
  • 2
    $\begingroup$ As I say in my answer, do not use t.test(). $\endgroup$
    – Noah
    Apr 24, 2023 at 4:59
  • $\begingroup$ I see! Thanks for your comments! Those are very helpful! $\endgroup$
    – Buffoon
    Apr 24, 2023 at 5:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.