Return to Question

The degrees of freedom are reflecting the number of observations of the HR and SC data, not the IOS data which should be around 90.

The degrees of freedom are reflecting the number of observations of the HR and SC data, not the IOS data which should be around 90.

IFI_ALL <- 
lmer(IF_interact ~ cond_id*GoldMSI_MT*(HR + SC + ztime + IOS) +
       (1|case_id:group_id)+
       (1|stim_id)+
       (1|instrument)+
       (1|notasi),
     data= comb_df)

> head(comb_df[,c(1,3,5,10,17,18,19,20)])


A tibble: 6 × 8
  case_id ztime cond_id IF_interact IOS[,1] section    HR    SC
  <fct>   <dbl> <fct>         <dbl>   <dbl> <chr>   <dbl> <dbl>
1 1        2.59 Notated       0.837   0.533 1          NA 0.256
2 1        2.59 Notated       0.837   0.533 2          NA 0.539
3 1        2.59 Notated       0.837   0.533 3          NA 0.378
4 1        2.59 Notated       0.837   0.533 4          NA 0.148
5 1        2.59 Notated       0.837   0.533 5          NA 0.117
6 1        2.59 Notated       0.837   0.533 6          NA 0.449

I'm trying to run posthocs for the three-way interactions using emtrends.

etsSC_C <- 
    emtrends(IFI_ALL, ~ cond_id|GoldMSI_MT, var = "SC",cov.reduce=meanpm1sd, 
             infer = c(T,T,T), adjust= "BH")
  PH_SC_C<- as.data.frame(contrast(etsSC_C, "pairwise", adjust= "BH"))

GoldMSI_MT = -0.4693676:
 contrast               estimate   SE      df t.ratio p.value
 Improvised - Memorised     0.35 0.10 1391.79   3.540  0.0012
 Improvised - Notated       0.16 0.08 1387.30   2.030  0.0458
 Memorised - Notated       -0.19 0.09 1394.13  -2.000  0.0458

GoldMSI_MT =  0.3168349:
 contrast               estimate   SE      df t.ratio p.value
 Improvised - Memorised     0.04 0.07 1388.35   0.560  0.8617
 Improvised - Notated      -0.01 0.06 1386.51  -0.120  0.9075
 Memorised - Notated       -0.04 0.06 1389.47  -0.750  0.8617

GoldMSI_MT =  1.1030373:
 contrast               estimate   SE      df t.ratio p.value
 Improvised - Memorised    -0.28 0.09 1387.38  -3.150  0.0051
 Improvised - Notated      -0.18 0.08 1387.02  -2.180  0.0446
 Memorised - Notated        0.10 0.07 1387.65   1.470  0.1413

Results are averaged over the levels of: HR, SC, ztime, IOS 
Degrees-of-freedom method: kenward-roger 
P value adjustment: BH method for 3 tests 

   etsIOS_C <- 
     emtrends(IFI_ALL, ~ cond_id|GoldMSI_MT, var = "IOS",cov.reduce=meanpm1sd, 
              infer = c(T,T,T), adjust= "BH")
   PH_IOS_C<-as.data.frame(contrast(etsIOS_C, "pairwise", adjust= "BH"))

GoldMSI_MT = -0.4693676:
 contrast               estimate   SE      df t.ratio p.value
 Improvised - Memorised     0.89 0.05 1399.71  16.770  <.0001
 Improvised - Notated       0.41 0.05 1419.48   7.930  <.0001
 Memorised - Notated       -0.48 0.05 1409.22  -9.010  <.0001

GoldMSI_MT =  0.3168349:
 contrast               estimate   SE      df t.ratio p.value
 Improvised - Memorised     0.37 0.04 1417.97   9.010  <.0001
 Improvised - Notated       0.03 0.03 1413.51   0.890  0.3723
 Memorised - Notated       -0.35 0.03 1413.39 -11.520  <.0001

GoldMSI_MT =  1.1030373:
 contrast               estimate   SE      df t.ratio p.value
 Improvised - Memorised    -0.14 0.07 1413.82  -1.920  0.0546
 Improvised - Notated      -0.36 0.05 1414.21  -6.610  <.0001
 Memorised - Notated       -0.22 0.04 1411.25  -4.890  <.0001

Results are averaged over the levels of: HR, SC, ztime, IOS 
Degrees-of-freedom method: kenward-roger 
P value adjustment: BH method for 3 tests

The degrees of freedom are reflecting the number of observations of the HR and SC data, not the IOS data which should be around 90.

I know this may have to do with the interactions in the original model. Are all of my assumptions here correct, and is there a solution, without using distinct() and running separate models? I've thought about looking into ref_grid and adding further arguments to cov.reduce but nothing seems to work.

Thank you in advance!

IFI_ALL <- 
lmer(IF_interact ~ cond_id*GoldMSI_MT*(HR + SC + ztime + IOS) +
       (1|case_id:group_id) +
       (1|stim_id) +
       (1|instrument) +
       (1|notasi),
     data= comb_df)

> head(comb_df[,c(1,3,5,10,17,18,19,20)])
 
A tibble: 6 × 8
  case_id ztime cond_id IF_interact IOS[,1] section    HR    SC
  <fct>   <dbl> <fct>         <dbl>   <dbl> <chr>   <dbl> <dbl>
1 1        2.59 Notated       0.837   0.533 1          NA 0.256
2 1        2.59 Notated       0.837   0.533 2          NA 0.539
3 1        2.59 Notated       0.837   0.533 3          NA 0.378
4 1        2.59 Notated       0.837   0.533 4          NA 0.148
5 1        2.59 Notated       0.837   0.533 5          NA 0.117
6 1        2.59 Notated       0.837   0.533 6          NA 0.449

I'm trying to run posthocs for the three-way interactions using emtrends.

etsSC_C <- 
    emtrends(IFI_ALL, ~ cond_id|GoldMSI_MT, 
    var =   "SC", cov.reduce=meanpm1sd, 
             infer = c(T,T,T), adjust= "BH")
  PH_SC_C<- as.data.frame(contrast(etsSC_C, "pairwise", adjust= "BH"))

GoldMSI_MT = -0.4693676:
 contrast               estimate   SE      df t.ratio p.value
 Improvised - Memorised     0.35 0.10 1391.79   3.540  0.0012
 Improvised - Notated       0.16 0.08 1387.30   2.030  0.0458
 Memorised - Notated       -0.19 0.09 1394.13  -2.000  0.0458

GoldMSI_MT =  0.3168349:
 contrast               estimate   SE      df t.ratio p.value
 Improvised - Memorised     0.04 0.07 1388.35   0.560  0.8617
 Improvised - Notated      -0.01 0.06 1386.51  -0.120  0.9075
 Memorised - Notated       -0.04 0.06 1389.47  -0.750  0.8617

GoldMSI_MT =  1.1030373:
 contrast               estimate   SE      df t.ratio p.value
 Improvised - Memorised    -0.28 0.09 1387.38  -3.150  0.0051
 Improvised - Notated      -0.18 0.08 1387.02  -2.180  0.0446
 Memorised - Notated        0.10 0.07 1387.65   1.470  0.1413

Results are averaged over the levels of: HR, SC, ztime, IOS 
Degrees-of-freedom method: kenward-roger 
P value adjustment: BH method for 3 tests 

etsIOS_C <- emtrends(IFI_ALL, ~ cond_id|GoldMSI_MT, var = "IOS", 
    cov.reduce=meanpm1sd, infer = c(T,T,T), adjust= "BH")
PH_IOS_C<-as.data.frame(contrast(etsIOS_C, "pairwise", adjust= "BH"))
        
        GoldMSI_MT = -0.4693676:
         contrast               estimate   SE      df t.ratio p.value
         Improvised - Memorised     0.89 0.05 1399.71  16.770  <.0001
         Improvised - Notated       0.41 0.05 1419.48   7.930  <.0001
         Memorised - Notated       -0.48 0.05 1409.22  -9.010  <.0001
        
        GoldMSI_MT =  0.3168349:
         contrast               estimate   SE      df t.ratio p.value
         Improvised - Memorised     0.37 0.04 1417.97   9.010  <.0001
         Improvised - Notated       0.03 0.03 1413.51   0.890  0.3723
         Memorised - Notated       -0.35 0.03 1413.39 -11.520  <.0001
        
        GoldMSI_MT =  1.1030373:
         contrast               estimate   SE      df t.ratio p.value
         Improvised - Memorised    -0.14 0.07 1413.82  -1.920  0.0546
         Improvised - Notated      -0.36 0.05 1414.21  -6.610  <.0001
         Memorised - Notated       -0.22 0.04 1411.25  -4.890  <.0001
    
        Results are averaged over the levels of: HR, SC, ztime, IOS 
        Degrees-of-freedom method: kenward-roger 
        P value adjustment: BH method for 3 tests

The degrees of freedom are reflecting the number of observations of the HR and SC data, not the IOS data which should be around 90.

I know this may have to do with the interactions in the original model. Are all of my assumptions here correct, and is there a solution, without using distinct() and running separate models? I've thought about looking into ref_grid and adding further arguments to cov.reduce but nothing seems to work.

I have a fairly complex linear mixed effect model in R and I believe that there are issues to do with repeated data when running my post hoc tests. The model formula is as follows:

etsSC_C <- 
    emtrends(IFI_ALL, ~ cond_id|GoldMSI_MT, var = "SC",cov.reduce=meanpm1sd, 
             infer = c(T,T,T), adjust= "BH")
  PH_SC_C<- as.data.frame(contrast(etsSC_C, "pairwise", adjust= "BH"))

I've realised that while the contrasts for SC and HR effects are perfectly fine, the p-values when I'm running contrasts for ztime and IOS are likely to be unreliable/inflated due to the amount of duplicated rows.

Is this correct, and is there a solution, without using distinct() and running separate models?

Edited to clarify further 23/04/24:

I have a fairly complex linear mixed effect model in R and I believe that there are issues to do with repeated data when running my post hoc tests. The model formula is as follows:

etsSC_C <- 
    emtrends(IFI_ALL, ~ cond_id|GoldMSI_MT, var = "SC",cov.reduce=meanpm1sd, 
             infer = c(T,T,T), adjust= "BH")
  PH_SC_C<- as.data.frame(contrast(etsSC_C, "pairwise", adjust= "BH"))

GoldMSI_MT = -0.4693676:
 contrast               estimate   SE      df t.ratio p.value
 Improvised - Memorised     0.35 0.10 1391.79   3.540  0.0012
 Improvised - Notated       0.16 0.08 1387.30   2.030  0.0458
 Memorised - Notated       -0.19 0.09 1394.13  -2.000  0.0458

GoldMSI_MT =  0.3168349:
 contrast               estimate   SE      df t.ratio p.value
 Improvised - Memorised     0.04 0.07 1388.35   0.560  0.8617
 Improvised - Notated      -0.01 0.06 1386.51  -0.120  0.9075
 Memorised - Notated       -0.04 0.06 1389.47  -0.750  0.8617

GoldMSI_MT =  1.1030373:
 contrast               estimate   SE      df t.ratio p.value
 Improvised - Memorised    -0.28 0.09 1387.38  -3.150  0.0051
 Improvised - Notated      -0.18 0.08 1387.02  -2.180  0.0446
 Memorised - Notated        0.10 0.07 1387.65   1.470  0.1413

Results are averaged over the levels of: HR, SC, ztime, IOS 
Degrees-of-freedom method: kenward-roger 
P value adjustment: BH method for 3 tests 

I've realised that while the contrasts for SC and HR effects are perfectly fine, the p-values when I'm running contrasts for ztime and IOS are likely to be unreliable/inflated due to the amount of duplicated rows:

   etsIOS_C <- 
     emtrends(IFI_ALL, ~ cond_id|GoldMSI_MT, var = "IOS",cov.reduce=meanpm1sd, 
              infer = c(T,T,T), adjust= "BH")
   PH_IOS_C<-as.data.frame(contrast(etsIOS_C, "pairwise", adjust= "BH"))

GoldMSI_MT = -0.4693676:
 contrast               estimate   SE      df t.ratio p.value
 Improvised - Memorised     0.89 0.05 1399.71  16.770  <.0001
 Improvised - Notated       0.41 0.05 1419.48   7.930  <.0001
 Memorised - Notated       -0.48 0.05 1409.22  -9.010  <.0001

GoldMSI_MT =  0.3168349:
 contrast               estimate   SE      df t.ratio p.value
 Improvised - Memorised     0.37 0.04 1417.97   9.010  <.0001
 Improvised - Notated       0.03 0.03 1413.51   0.890  0.3723
 Memorised - Notated       -0.35 0.03 1413.39 -11.520  <.0001

GoldMSI_MT =  1.1030373:
 contrast               estimate   SE      df t.ratio p.value
 Improvised - Memorised    -0.14 0.07 1413.82  -1.920  0.0546
 Improvised - Notated      -0.36 0.05 1414.21  -6.610  <.0001
 Memorised - Notated       -0.22 0.04 1411.25  -4.890  <.0001

Results are averaged over the levels of: HR, SC, ztime, IOS 
Degrees-of-freedom method: kenward-roger 
P value adjustment: BH method for 3 tests

The degrees of freedom are reflecting the number of observations of the HR and SC data, not the IOS data which should be around 90.

I know this may have to do with the interactions in the original model. Are all of my assumptions here correct, and is there a solution, without using distinct() and running separate models? I've thought about looking into ref_grid and adding further arguments to cov.reduce but nothing seems to work.