0
$\begingroup$

I am modeling data I described here. I have stopped treating clusters individually and have since averaged them, so now I have three experimental conditions, Type, Relevance, and Taught. Type has three levels, the others two. There were 37 subjects and each subject had 12 runs, or imaging sessions, with 30 trials per run. Since I've now divided the data in half and am using half the runs for cluster generation and have the runs for my brain-behavior regression, effectively each subject has six runs. Relevance and Taught were trial-level variables, while Type was a run-level variable. Brain activation (at least the way I am quantifying it, as the beta for a regressor in a subject level time series), can only be measured per run. On each trial there is also a behavioral response, accuracy.

I am regressing the contrast over levels of Relevance in brain activation onto Type, Taught, and the contrast over levels of Relevance in accuracy.

I first had

My model was std_brain ~ type * taught * rel_behavior + (1|subject/run).

But subject wasn't associated with any variance and the model generated a singularity warning, so I changed it to:

std_brain ~ type * taught * rel_behavior + (1|subject:run)

believing that would be sufficient to adjust the df to account for repeated measures over run (the two levels of Taught), and in fact the df seem the same for (1|subject:run) and (1|subject/run).

I get a non-singular result, but when I inspect the residuals, there are some extreme points and some rightward skew, so I ran the same model with robustlmm's rlmer. I get similar results (generating significance tests with the method described here).

Here is my question: it seems like robustlmm just doesn't give you singularity warnings, because when I look at the output, I don't get the warning, but the variance associated with subject:run is back to being 0:

Robust linear mixed model fit by DAStau 
Formula: std_brain ~ rel_behavior * type * taught + (1 | subject:run) 
   Data: avg_odd_data_dx 

Scaled residuals: 
    Min      1Q  Median      3Q     Max 
-6.2791 -0.5709 -0.0391  0.5684 12.4567 

Random effects:
 Groups      Name        Variance Std.Dev.
 subject:run (Intercept) 0.0000   0.0000  
 Residual                0.4227   0.6501  
Number of obs: 444, groups: subject:run, 222

Fixed effects:
                           Estimate Std. Error t value
(Intercept)                 0.13062    0.07905   1.652
rel_behavior                0.20126    0.09248   2.176
typeN                      -0.15978    0.11420  -1.399
typeY                      -0.19014    0.11412  -1.666
taughtU                    -0.29242    0.12096  -2.418
rel_behavior:typeN         -0.29149    0.12109  -2.407
rel_behavior:typeY         -0.31456    0.13745  -2.289
rel_behavior:taughtU       -0.32683    0.15005  -2.178
typeN:taughtU               0.31981    0.17336   1.845
typeY:taughtU               0.38225    0.17323   2.207
rel_behavior:typeN:taughtU  0.36639    0.20958   1.748
rel_behavior:typeY:taughtU  0.51220    0.20950   2.445

Correlation of Fixed Effects:
            (Intr) rl_bhv typeN  typeY  taghtU rl_b:N rl_b:Y rl_b:U typN:U typY:U r_:N:U
rel_behavir -0.159                                                                      
typeN       -0.692  0.110                                                               
typeY       -0.693  0.110  0.480                                                        
taughtU     -0.654  0.104  0.452  0.453                                                 
rl_bhvr:tyN  0.121 -0.764 -0.257 -0.084 -0.079                                          
rl_bhvr:tyY  0.107 -0.673 -0.074 -0.243 -0.070  0.514                                   
rl_bhvr:tgU  0.098 -0.616 -0.068 -0.068 -0.376  0.471  0.415                            
typeN:tghtU  0.456 -0.072 -0.659 -0.316 -0.698  0.170  0.049  0.262                     
typeY:tghtU  0.456 -0.073 -0.316 -0.659 -0.698  0.055  0.160  0.262  0.487              
rl_bhvr:N:U -0.070  0.441  0.149  0.049  0.269 -0.578 -0.297 -0.716 -0.433 -0.188       
rl_bhvr:Y:U -0.070  0.441  0.049  0.160  0.269 -0.337 -0.656 -0.716 -0.188 -0.406  0.513

Robustness weights for the residuals: 
 347 weights are ~= 1. The remaining 97 ones are summarized as
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
  0.108   0.517   0.712   0.676   0.877   0.997 

Robustness weights for the random effects: 
 All 222 weights are ~= 1.

I have been getting so many singularity warnings for so long that it is making me uneasy to see that zero-variance random effect. Should I be concerned about this? What is more of a problem, skewed and extreme residuals in an lme4 linear mixed model or a zero variance random effect in a robustly estimated linear mixed model? One thing I have considered: given that I'm calculating p-values with estimated degrees of freedom from the model generated in lme4 anyway, would it be very strange to remove this last random effect, run a non-mixed model robust regression, but use the dfs from my model in lme4 to calculate p-values? It feels pretty strange.

Edit: As requested, here's the result if I run a model using lm with no random effects:

Call:
lm(formula = std_brain ~ rel_behavior * type * taught, data = avg_odd_data_dx)

Residuals:
    Min      1Q  Median      3Q     Max 
-4.0009 -0.4156 -0.0690  0.3723  7.8202 

Coefficients:
                           Estimate Std. Error t value Pr(>|t|)   
(Intercept)                  0.3447     0.1168   2.951  0.00334 **
rel_behavior                 0.1498     0.1366   1.096  0.27365   
typeN                       -0.4421     0.1687  -2.620  0.00911 **
typeY                       -0.3232     0.1686  -1.916  0.05596 . 
taughtU                     -0.5235     0.1787  -2.929  0.00358 **
rel_behavior:typeN          -0.2798     0.1789  -1.564  0.11857   
rel_behavior:typeY          -0.3991     0.2031  -1.965  0.05006 . 
rel_behavior:taughtU        -0.3070     0.2217  -1.385  0.16682   
typeN:taughtU                0.6532     0.2562   2.550  0.01112 * 
typeY:taughtU                0.4888     0.2560   1.909  0.05686 . 
rel_behavior:typeN:taughtU   0.3616     0.3097   1.168  0.24359   
rel_behavior:typeY:taughtU   0.7392     0.3096   2.388  0.01737 * 
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.9853 on 432 degrees of freedom
Multiple R-squared:  0.05326,   Adjusted R-squared:  0.02916 
F-statistic:  2.21 on 11 and 432 DF,  p-value: 0.0132

Edit 2: I've just realized I was wrong about robustlmm not doing anything regarding the isSingular warning. I went back and ran the robust model with my full set of random effects, (1 | subject/run), and I did get printed boundary (singular) fit: see ?isSingular with the two zero-variance random effects in the model -- it just doesn't print out the warning when you run summary. So I suppose I have another question, which is why doesn't it do that for the model whose output also had a zero-variance random effect?

$\endgroup$
2
  • $\begingroup$ It appears that you don't need random effects at all. Please post the summary output of lm(std_brain ~ rel_behavior * type * taught, data) $\endgroup$ Oct 6 '21 at 17:32
  • $\begingroup$ @RobertLong I added it. My problem is that then I don't know how to deal with the fact that I have repeated measures -- I have two levels of Taught per run and ~ 6 runs per subject. $\endgroup$
    – Katie
    Oct 6 '21 at 22:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.