3
$\begingroup$

Suppose the following model:

DV is reaction time. The predictor is a categorical factor with three levels, manipulated within participants. Each participant gets fifty trials at each level of this factor. In constructing a regression model, the factor is contrast coded such that Level A is compared to Level B, and Level B is compared to Level C.

The data are analyzed at the trial level, and I would like to include a random subject slope for this factor. lme4 fits a separate slope for each contrast (i.e., a slope for Level A vs Level B; a slope for Level B vs. Level C).

Now, suppose I use the method described in Bates et al. (2015, https://arxiv.org/abs/1506.04967) and determine that the random effects structure is overly complex given the data. Indeed, that there should only be two random subject terms. Further, assume that there is a healthy amount of variance for slope A vs B, but no variance for slope B vs C.

Is it possible and/or advisable to only include the random subject slope for the A vs B contrast, and not the B vs C contrast? Is this possible in lme4?

Edit: Here is some data showing this phenomenon. This is a bad example beacause there is a perfect correlation between the random effects. But, notice how much more variance there is in the slope for 2vs1 than 3vs1.

data$PPT <- as.factor(data$PPT)
data$Item <- as.factor(data$Item)
data$IV1 <- as.factor(data$IV1)

require(lme4)
m <- lmer(DV ~ IV1 + (1 + IV1|PPT) + (1|Item), data = data)
summary(m)

PPT,Item,IV1,DV
1,1,1,0.690418559
1,2,1,0.726320273
1,3,1,0.804200471
2,1,1,0.962554853
2,2,1,0.881709529
2,3,1,0.726924832
3,1,1,0.324313446
3,2,1,0.397593471
3,3,1,0.600267969
4,1,1,0.007660862
4,2,1,0.613661302
4,3,1,0.701654521
5,1,1,0.715478067
5,2,1,0.72973927
5,3,1,0.243646406
6,1,1,0.102868704
6,2,1,0.835788904
6,3,1,0.879894264
7,1,1,0.646464624
7,2,1,0.290197203
7,3,1,0.468700931
8,1,1,0.300354425
8,2,1,0.386144582
8,3,1,0.336186196
9,1,1,0.762558837
9,2,1,0.262034368
9,3,1,0.810487946
10,1,1,0.72507343
10,2,1,0.103301606
10,3,1,0.819375795
11,1,1,3.865723501
11,2,1,3.282892197
11,3,1,3.718256113
12,1,1,3.799044616
12,2,1,3.007044599
12,3,1,3.556554396
13,1,1,3.66787012
13,2,1,3.417308436
13,3,1,3.496663292
14,1,1,7.219547568
14,2,1,7.390538962
14,3,1,7.845634649
15,1,1,10.60401189
15,2,1,10.75755367
15,3,1,10.19106193
16,1,1,4.646229035
16,2,1,-0.570717613
16,3,1,2.685347747
17,1,1,-0.902583315
17,2,1,3.826389671
17,3,1,2.429604901
18,1,1,-0.444543923
18,2,1,-0.563763378
18,3,1,5.773696849
19,1,1,8.715439183
19,2,1,8.729044918
19,3,1,8.960825012
20,1,1,10.58463501
20,2,1,10.00071435
20,3,1,10.1160007
21,1,1,2.958587038
21,2,1,2.253147922
21,3,1,2.169195933
22,1,1,2.035884448
22,2,1,2.173522198
22,3,1,2.587967874
23,1,1,4.105819445
23,2,1,4.659031049
23,3,1,4.008122345
24,1,1,4.532558271
24,2,1,4.677689434
24,3,1,4.678275241
25,1,1,6.962798621
25,2,1,6.915245539
25,3,1,6.948560771
26,1,1,6.64033688
26,2,1,6.47252625
26,3,1,6.662764257
27,1,1,6.195048749
27,2,1,6.600919889
27,3,1,6.30772784
28,1,1,3.278737289
28,2,1,5.055771867
28,3,1,8.586051713
29,1,1,2.286495723
29,2,1,2.160040414
29,3,1,5.145300269
30,1,1,4.10031788
30,2,1,8.756800138
30,3,1,6.915743005
31,1,1,7.429209
31,2,1,1.159563
31,3,1,-5.634155
32,1,1,-4.482404
32,2,1,3.725128
32,3,1,6.376419
33,1,1,6.367126
33,2,1,-1.922212
33,3,1,0.949954
34,1,1,12.46231881
34,2,1,10.87718438
34,3,1,10.06357163
35,1,1,3.635709487
35,2,1,8.726583385
35,3,1,10.92620501
36,1,1,11.76357695
36,2,1,4.159483477
36,3,1,12.58759115
37,1,1,11.75182867
37,2,1,11.08389517
37,3,1,12.38644649
38,1,1,7.877327895
38,2,1,4.460531683
38,3,1,6.602453436
39,1,1,8.537950631
39,2,1,12.75788546
39,3,1,9.239483058
40,1,1,7.50918545
40,2,1,3.099585385
40,3,1,11.08225114
1,4,2,3.544108799
1,5,2,3.153470211
1,6,2,3.958081556
2,4,2,3.218065143
2,5,2,3.281335494
2,6,2,3.685038505
3,4,2,3.613528222
3,5,2,3.250383774
3,6,2,3.718901425
4,4,2,3.415384855
4,5,2,3.782321218
4,6,2,3.442750593
5,4,2,7.178009506
5,5,2,7.388086919
5,6,2,7.367573597
6,4,2,7.315038344
6,5,2,7.791804979
6,6,2,7.397225331
7,4,2,7.195581405
7,5,2,7.791296004
7,6,2,7.993662942
8,4,2,2.941572757
8,5,2,2.646572263
8,6,2,2.646177944
9,4,2,2.414143449
9,5,2,2.244883718
9,6,2,2.105037823
10,4,2,2.942570413
10,5,2,2.646001063
10,6,2,2.827564998
11,4,2,0.004163077
11,5,2,0.56469036
11,6,2,1.013952811
12,4,2,9.722891457
12,5,2,1.089671074
12,6,2,8.592893618
13,4,2,8.853141748
13,5,2,9.842814883
13,6,2,8.820564101
14,4,2,0.912985906
14,5,2,2.527697507
14,6,2,1.427506127
15,4,2,0.231090639
15,5,2,6.794761619
15,6,2,3.661965587
16,4,2,2.25853418
16,5,2,7.881714289
16,6,2,5.97705908
17,4,2,5.872285076
17,5,2,2.513927458
17,6,2,1.530875578
18,4,2,3.767898127
18,5,2,2.274031825
18,6,2,5.512919424
19,4,2,8.92774853
19,5,2,6.999869472
19,6,2,5.202125908
20,4,2,9.088142604
20,5,2,4.510608503
20,6,2,8.063301435
21,4,2,3.807087468
21,5,2,3.416195333
21,6,2,3.804771969
22,4,2,3.601193128
22,5,2,3.869713995
22,6,2,3.939390889
23,4,2,3.413930655
23,5,2,3.340439814
23,6,2,3.135196464
24,4,2,3.727491036
24,5,2,3.676463089
24,6,2,3.973719189
25,4,2,7.251830996
25,5,2,7.399370652
25,6,2,7.189077173
26,4,2,7.33059826
26,5,2,7.022480222
26,6,2,7.488514261
27,4,2,7.498971958
27,5,2,7.363936754
27,6,2,7.906968396
28,4,2,2.144851207
28,5,2,2.334106942
28,6,2,2.546860926
29,4,2,2.70686534
29,5,2,2.699632662
29,6,2,2.418149412
30,4,2,2.449250496
30,5,2,2.798519022
30,6,2,2.037459216
31,4,2,6.02716533
31,5,2,2.948682545
31,6,2,2.620238835
32,4,2,5.681633142
32,5,2,5.66457844
32,6,2,8.978847636
33,4,2,6.538479467
33,5,2,1.28196303
33,6,2,3.664491028
34,4,2,4.599649075
34,5,2,6.862497476
34,6,2,6.135916112
35,4,2,3.226242183
35,5,2,1.815466546
35,6,2,4.313277666
36,4,2,9.989533491
36,5,2,9.714314019
36,6,2,8.409759534
37,4,2,0.154394068
37,5,2,4.06040419
37,6,2,9.853443312
38,4,2,8.579243716
38,5,2,6.554535878
38,6,2,1.858351233
39,4,2,2.969111478
39,5,2,9.188567221
39,6,2,4.283672771
40,4,2,8.309907866
40,5,2,2.4368261
40,6,2,2.79011029
1,7,3,3.944245076
1,8,3,3.645605102
1,9,3,3.251908288
2,7,3,3.151946475
2,8,3,3.398600512
2,9,3,3.604361614
3,7,3,3.876747581
3,8,3,3.615641563
3,9,3,3.636755516
4,7,3,3.229502309
4,8,3,3.650286024
4,9,3,3.949985561
5,7,3,3.649669532
5,8,3,3.211305094
5,9,3,3.986169534
6,7,3,3.722927312
6,8,3,3.804507995
6,9,3,3.729109499
7,7,3,3.039859904
7,8,3,3.049121278
7,9,3,3.33101854
8,7,3,3.034075165
8,8,3,3.072159815
8,9,3,3.927118012
9,7,3,3.821984188
9,8,3,3.56538544
9,9,3,3.205599937
10,7,3,3.738261709
10,8,3,3.996893544
10,9,3,3.693912206
11,7,3,5.340076467
11,8,3,9.276143445
11,9,3,2.767432473
12,7,3,10.21602579
12,8,3,4.592533068
12,9,3,10.94523958
13,7,3,7.948683929
13,8,3,8.42496413
13,9,3,3.863686907
14,7,3,10.21993536
14,8,3,5.482294249
14,9,3,10.73275057
15,7,3,8.930422456
15,8,3,3.694551607
15,9,3,9.350531994
16,7,3,9.56150288
16,8,3,5.174201737
16,9,3,5.554632802
17,7,3,5.077996559
17,8,3,9.680018937
17,9,3,5.814166791
18,7,3,5.488075068
18,8,3,10.86563455
18,9,3,6.259021065
19,7,3,11.56035022
19,8,3,3.650666495
19,9,3,5.826988583
20,7,3,9.192700853
20,8,3,11.50871423
20,9,3,10.51302419
21,7,3,2.144509917
21,8,3,3.165764756
21,9,3,0.859654043
22,7,3,2.105697187
22,8,3,8.63992964
22,9,3,8.245907673
23,7,3,9.092851901
23,8,3,1.322559052
23,9,3,3.031627714
24,7,3,9.037750575
24,8,3,6.309450611
24,9,3,2.749860122
25,7,3,2.875130866
25,8,3,8.077022168
25,9,3,4.873940193
26,7,3,2.866258266
26,8,3,0.507409107
26,9,3,7.113317024
27,7,3,7.982322854
27,8,3,2.684196336
27,9,3,1.07567063
28,7,3,9.014605454
28,8,3,9.693430202
28,9,3,3.448750676
29,7,3,9.142211399
29,8,3,9.762733314
29,9,3,8.227290314
30,7,3,9.932851498
30,8,3,4.429624636
30,9,3,8.022678366
31,7,3,6.568761185
31,8,3,7.552628466
31,9,3,3.771688989
32,7,3,9.028813365
32,8,3,1.076440331
32,9,3,5.154056529
33,7,3,8.818667352
33,8,3,2.06485883
33,9,3,2.860083318
34,7,3,5.70214451
34,8,3,3.124169262
34,9,3,0.230815596
35,7,3,9.383389027
35,8,3,1.157133631
35,9,3,2.549956691
36,7,3,1.009046579
36,8,3,3.354598416
36,9,3,3.631063405
37,7,3,3.194663187
37,8,3,7.731237512
37,9,3,5.742790401
38,7,3,2.434049727
38,8,3,8.306045918
38,9,3,8.898874417
39,7,3,4.313672658
39,8,3,9.306310168
39,9,3,6.243225695
40,7,3,2.295790969
40,8,3,3.111754654
40,9,3,7.313299176
$\endgroup$
4
  • 1
    $\begingroup$ +1`You could also consider adding some R code similar to what you posted on SO earlier :) ...in order to make an actual reproducible example (which is highly recommended)...try to make the example demonstrate the question you are posing, rather than saying "suppose it turned out that...." $\endgroup$ Commented Nov 9, 2018 at 21:43
  • $\begingroup$ Tried to do so, but had difficulty generating data that didn't have a perfect correlation between random effects. Nevertheless, this shows what I mean regarding the different variances in slopes. $\endgroup$
    – Dave
    Commented Nov 9, 2018 at 23:31
  • $\begingroup$ To generate correlated random slopes, use MASS::mvrnorm. If you want the random slope variable to vary within each case, then generate data that varies within case and add it to the random slope to create a new variable that is composed of the random slope and the regular variable. $\endgroup$ Commented Nov 10, 2018 at 5:54
  • $\begingroup$ @HeteroskedasticJim No it's arxiv.org/abs/1506.04967 $\endgroup$
    – Dave
    Commented Nov 11, 2018 at 0:47

1 Answer 1

3
$\begingroup$

Is it possible? I think so, if you manually define two dummy variables as follows:

DummyA = 1 if IV1 = A and DummyA = 0 otherwise

DummyC = 1 if IV1 = C and DummyC = 0 otherwise

With this coding, IV1 = B when DummyA = 0 and DummyC = 0.

Then include the two dummy variables in your model like this:

require(lme4)
m1 <- lmer(DV ~ DummyA + DummyC + (1 + DummyA|PPT) + (1|Item), 
          data = data)
summary(m1)

Is it advisable? I guess the concern is that the model

m2 <- lmer(DV ~ DummyA + DummyC + (1 + DummyC|PPT) + (1|Item), 
          data = data)
summary(m2)

could also be supported by the data. What makes you believe that it's more appropriate to fit model m1 to the data rather than model m2? For example, did you look at something like conditional R-squared to assess which of the two models should be preferred?

The reality is that you should most likely use random effects for both dummy variables but your data just can't support this. So be prepared to not use random effects for either and report that your model didn't support random effects for both at the same time, but did support random effects for one at a time.

Or perhaps compare the conditional R squared of the model without random slopes for the two dummies against the conditional R squared of the models with a random slope for one dummy at a time. How much of an improvement in the conditional R squared you achieve by including a random slope for one dummy? Is the improvement sizeable or negligible? If it is sizeable for each of the two dummies, perhaps that suggests that both should have random slopes (though that is not supported by your data). To just allow one to have a random slope essentially becomes an arbitrary decision. But if one dummy shows a sizeable improvement and the other a negligible improvement, you could feel more confident about allowing the dummy with a sizeable improvement to have a random slope. If both dummies show negligible improvement, then arguably there is no need to include random slopes for either.

$\endgroup$
3
  • 1
    $\begingroup$ Hi Isabella, thank you for the reply. I'd determined that (for instance) m2 is preferable by looking at the amount of variance associated with each slope when they are both fit. After using PCA on the random effects structure, it was determined that only two random subject effects should be fit. These two factors together suggest that a slope for only one contrast should be fit, and which it should be (based on variance associated with each). $\endgroup$
    – Dave
    Commented Nov 20, 2018 at 0:51
  • $\begingroup$ Thank you very much for your update, @Dave! I am not familiar with the use of PCA on the random effects structure - can you give me a reference for it? As long as you can clearly justify your choice of which contrast is best supported by the data, you should be fine. $\endgroup$ Commented Nov 20, 2018 at 2:16
  • 1
    $\begingroup$ Hi Isabella, here is a link to the paper! arxiv.org/abs/1506.04967 Thanks once again. $\endgroup$
    – Dave
    Commented Nov 20, 2018 at 18:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.