2
$\begingroup$

I'm a little puzzled because this has never happened to me before when using the tab_model function - I'm writing up the results for a paper, and when I use tab_model() from the sjPlot package (to obtain an html, nicer looking table) I've found that some of the results, especifically the p-values, change drastically between the two functions. The model was:

m.1 = lmer(EMG ~ Time_sd* Cond.num + EDA_cs + Arousal+ (1+Time_sd* Cond.num|SUBJECT) + (1+Time_sd |Video), data=nonadata, REML=TRUE, control = lmerControl(optimizer="bobyqa", optCtrl=list(maxfun=2e5)))

Output:

Formula: EMG_10000 ~ Time_sd * Cond.num + EDA_cs + Arousal + (1 + Time_sd *  
    Cond.num | SUBJECT) + (1 + Time_sd | Video)
   Data: nonadata
Control: lmerControl(optimizer = "bobyqa", optCtrl = list(maxfun = 2e+05))

REML criterion at convergence: 168357.1

Scaled residuals: 
     Min       1Q   Median       3Q      Max 
-15.7964  -0.2780  -0.0459   0.2062  22.0285 

Random effects:
 Groups   Name             Variance Std.Dev. Corr             
 SUBJECT  (Intercept)       6.0407  2.4578                    
          Time_sd           1.5212  1.2334    0.58            
          Cond.num         20.9494  4.5771   -0.57 -0.51      
          Time_sd:Cond.num  3.3021  1.8172   -0.38 -0.39  0.49
 Video    (Intercept)       1.6525  1.2855                    
          Time_sd           0.1237  0.3516   0.83             
 Residual                  75.6901  8.7000                    
Number of obs: 23425, groups:  SUBJECT, 48; Video, 8

Fixed effects:
                   Estimate Std. Error         df t value Pr(>|t|)    
(Intercept)      -6.307e-01  6.268e-01  1.952e+01  -1.006  0.32663    
Time_sd          -6.996e-01  2.252e-01  2.524e+01  -3.106  0.00464 ** 
Cond.num          1.279e+00  1.131e+00  1.310e+01   1.130  0.27861    
EDA_cs            2.776e-01  6.152e-02  1.357e+04   4.513 6.45e-06 ***
Arousal           3.042e-02  5.338e-02  8.022e+03   0.570  0.56879    
Time_sd:Cond.num  8.496e-01  3.805e-01  1.515e+01   2.233  0.04104 *  
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

but tab_model(m.1):

enter image description here

For example, the Time_sd x Cond.num interaction went from p =.04 to p=.026. Any idea of what is possibly going on? The parameter estimates don't seem to change, only the p-values.

Improve this question
$\endgroup$
12
  • 1
    $\begingroup$ What package do you use to fit the mixed model? Possibly nlme together with lmerTest? I suspect the difference comes from the different methods to calculate the degrees of freedom. tab_model uses the Wald method by default whereas lmerTest uses Satterthwaite. What do you get if you use tab_model(m1, df.method = "satterthwaite", show.df = TRUE)? $\endgroup$
    – COOLSerdash
    Commented Jun 15, 2021 at 16:04
  • $\begingroup$ I use lme4; I tried your suggestion and I get the following error message. But yes, I think you're right, the issue has to do with how tab_model() calculates the dfs. tab_model(m.1, df.method = "satterthwaite", show.df = TRUE) Error in if (fam.info$is_linear) transform <- NULL else transform <- "exp" : argument is of length zero In addition: Warning message: Could not access model information. $\endgroup$
    – Luminosa
    Commented Jun 15, 2021 at 16:15
  • $\begingroup$ And when I try p.val="kr", for Kenward Rogers, I get the following error: Error in h(simpleError(msg, call)) : error in evaluating the argument 'x' in selecting a method for function 'forceSymmetric': cannot allocate vector of size 4.1 Gb I suspect my dataset is too big for tab_model()? I have 23405 dfs... $\endgroup$
    – Luminosa
    Commented Jun 15, 2021 at 16:17
  • 1
    $\begingroup$ I meant the function call of the model itself (lmer I presume). And please add this to your question, not the comments. $\endgroup$
    – COOLSerdash
    Commented Jun 15, 2021 at 16:45
  • 1
    $\begingroup$ "my field (media psychology) does not really let you publish without a p-value" ! Then I'm sorry to tell you that you're in the wrong field. I would recommend that you explain why you chose Satterthwaite's approximation, rather than say, Kenward-Rogers or any other arbitrary choice. Note that lme4::lmer does not produce p-values for mixed models, and that's for very good reasons. I assume you are using something like lmertest instead. $\endgroup$
    – Robert Long
    Commented Jun 15, 2021 at 20:24

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.