Edited to clarify further 23/04/24:
I have a fairly complex linear mixed effect model in R and I believe that there are issues to do with repeated data when running my post hoc tests. The model formula is as follows:
IFI_ALL <-
lmer(IF_interact ~ cond_id*GoldMSI_MT*(HR + SC + ztime + IOS) +
(1|case_id:group_id) +
(1|stim_id) +
(1|instrument) +
(1|notasi),
data= comb_df)
There are theoretical reasons why all should be in the same model, and my model fit is good. Within the dataset, the fixed effects have different levels of repetitions, whereby GoldMSI_MT has one value per participant, ztime and IOS repeat across the 3 cond_ids per participant, and HR and SC are longer time series physiological correlation data, with some missing values.
Here is an example of a portion of my dataset:
> head(comb_df[,c(1,3,5,10,17,18,19,20)])
A tibble: 6 × 8
case_id ztime cond_id IF_interact IOS[,1] section HR SC
<fct> <dbl> <fct> <dbl> <dbl> <chr> <dbl> <dbl>
1 1 2.59 Notated 0.837 0.533 1 NA 0.256
2 1 2.59 Notated 0.837 0.533 2 NA 0.539
3 1 2.59 Notated 0.837 0.533 3 NA 0.378
4 1 2.59 Notated 0.837 0.533 4 NA 0.148
5 1 2.59 Notated 0.837 0.533 5 NA 0.117
6 1 2.59 Notated 0.837 0.533 6 NA 0.449
I'm trying to run posthocs for the three-way interactions using emtrends.
etsSC_C <-
emtrends(IFI_ALL, ~ cond_id|GoldMSI_MT,
var = "SC", cov.reduce=meanpm1sd,
infer = c(T,T,T), adjust= "BH")
PH_SC_C<- as.data.frame(contrast(etsSC_C, "pairwise", adjust= "BH"))
GoldMSI_MT = -0.4693676:
contrast estimate SE df t.ratio p.value
Improvised - Memorised 0.35 0.10 1391.79 3.540 0.0012
Improvised - Notated 0.16 0.08 1387.30 2.030 0.0458
Memorised - Notated -0.19 0.09 1394.13 -2.000 0.0458
GoldMSI_MT = 0.3168349:
contrast estimate SE df t.ratio p.value
Improvised - Memorised 0.04 0.07 1388.35 0.560 0.8617
Improvised - Notated -0.01 0.06 1386.51 -0.120 0.9075
Memorised - Notated -0.04 0.06 1389.47 -0.750 0.8617
GoldMSI_MT = 1.1030373:
contrast estimate SE df t.ratio p.value
Improvised - Memorised -0.28 0.09 1387.38 -3.150 0.0051
Improvised - Notated -0.18 0.08 1387.02 -2.180 0.0446
Memorised - Notated 0.10 0.07 1387.65 1.470 0.1413
Results are averaged over the levels of: HR, SC, ztime, IOS
Degrees-of-freedom method: kenward-roger
P value adjustment: BH method for 3 tests
I've realised that while the contrasts for SC and HR effects are perfectly fine, the p-values when I'm running contrasts for ztime and IOS are likely to be unreliable/inflated due to the amount of duplicated rows:
etsIOS_C <- emtrends(IFI_ALL, ~ cond_id|GoldMSI_MT, var = "IOS",
cov.reduce=meanpm1sd, infer = c(T,T,T), adjust= "BH")
PH_IOS_C<-as.data.frame(contrast(etsIOS_C, "pairwise", adjust= "BH"))
GoldMSI_MT = -0.4693676:
contrast estimate SE df t.ratio p.value
Improvised - Memorised 0.89 0.05 1399.71 16.770 <.0001
Improvised - Notated 0.41 0.05 1419.48 7.930 <.0001
Memorised - Notated -0.48 0.05 1409.22 -9.010 <.0001
GoldMSI_MT = 0.3168349:
contrast estimate SE df t.ratio p.value
Improvised - Memorised 0.37 0.04 1417.97 9.010 <.0001
Improvised - Notated 0.03 0.03 1413.51 0.890 0.3723
Memorised - Notated -0.35 0.03 1413.39 -11.520 <.0001
GoldMSI_MT = 1.1030373:
contrast estimate SE df t.ratio p.value
Improvised - Memorised -0.14 0.07 1413.82 -1.920 0.0546
Improvised - Notated -0.36 0.05 1414.21 -6.610 <.0001
Memorised - Notated -0.22 0.04 1411.25 -4.890 <.0001
Results are averaged over the levels of: HR, SC, ztime, IOS
Degrees-of-freedom method: kenward-roger
P value adjustment: BH method for 3 tests
The degrees of freedom are reflecting the number of observations of the HR and SC data, not the IOS data which should be around 90.
I know this may have to do with the interactions in the original model. Are all of my assumptions here correct, and is there a solution, without using distinct() and running separate models? I've thought about looking into ref_grid and adding further arguments to cov.reduce but nothing seems to work.
