1
$\begingroup$

I got stuck interpreting the result of a generalised linear mixed model (GLMM). Feedbacks on how to compare two coefficients within a categorical fixed effect would be really helpful!

To be specific, the research question I ask is that are mind-wandering minds more likely to lead to deliberate thought than at-present minds? So the response variable is deliberateness (1 or 0), the predictor variable is attention status (at-present vs. mind-wandering). I also wanted to include Participant ID and the activity (both categorical) as the random effects.

I used the GLMM model because: 1. the response was binary, 2. it was a repeated measure, each participant received this question 18 times. 3. there were random missing values.

I used the GLMM package in R, my code was:

intent_status <- glmm(
  deliberate ~ 0 + Status_Q, 
  random = list(~ 0 + Participant, ~ 0 + activity), 
  varcomps.names = c("Participant", "activity"), 
  data = intent_status, 
  family.glmm = bernoulli.glmm, m = 10^4, debug = TRUE)

The result is:

summary(intent_status)

Call:
glmm(fixed = deliberate ~ 0 + Status_Q, random = list(~0 + Participant, 
  ~0 + Day_Recons), varcomps.names = c("at-present", "mind-wandering"), 
  data = intent_status, family.glmm = bernoulli.glmm, m = 10^4, 
  debug = TRUE)

Link is: "logit (log odds)"

Fixed Effects:
                        Estimate Std. Error z value Pr(>|z|)    
Status_Qat-present       1.1898     0.1215   9.796   <2e-16 ***
Status_Qmind-wandering   0.2660     0.1303   2.042   0.0412 *  
 ---
 Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1


Variance Components for Random Effects (P-values are one-tailed):
                Estimate Std. Error z value Pr(>|z|)/2    
Participant      2.29233    0.27786   8.250    < 2e-16 ***
activity         0.26057    0.08107   3.214   0.000655 ***
 ---
 Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

From the p-values of fixe effects I know both coefficients of at-present and mind-wandering were significant (both B !=0). But how do i know if the B(at-present) significantly larger than B(mind-wandering)? I searched on-line but wasn't able to find the answer that I want.

Please let me know if my approach is sensible to the original question, which is "are mind-wandering minds more likely to lead to deliberate thought than at-present minds"?

$\endgroup$
0
$\begingroup$

The $p$-value doesn't tell you anything useful about the differences between deliberation among wandering versus at-present minds. Generally, because the $p$-value does not account for effect size and it is misinterpreted by everybody. But specifically for a GLMM, the $p$-values printed in the call to summary are calculated incorrectly. See ?confint.glmer.

You say both groups are "significant". B != 0 here only means the proportion of deliberation was different from 0.5 in both groups... so who cares? It is not "significant" in any sense of the word (difference from 0.5 was not a respecified hypothesis).

To directly compare the groups, suppress the 0+ from your formula object. This will estimate a log odds ratio comparing the groups. Then fit the random effects model which omits the terms. Compare them with anova. See ?glmm for examples of how that's done.

$\endgroup$
  • $\begingroup$ @ AdamO, thanks so much for the explanation! I should have looked into the R documents more closely. I suppressed the 0+ in my formula, and the intercept changed to "at-present", so now I have the direct comparison between two group, which is awesome. I went on using anova(full, null, test="Chisq") to compare the full model with the null model, but I encountered an error "Error in UseMethod('anova') : no applicable method for 'anova' applied to an object of class 'MCMCglmm'". I searched online for how to compare two glmm models in r, but couldn't find one. Any further instructions are helpful $\endgroup$ – potpot_g May 19 '18 at 17:15
  • $\begingroup$ @potpot_g sorry I was assuming you were using the lme4 package instead. I don't know how inference is done in this package. You'll have to look up that info elsewhere. There should be a way to compare nested models. Good luck! $\endgroup$ – AdamO May 20 '18 at 2:10
  • $\begingroup$ you're already really informative on my misunderstanding of p-value for GLMM! I'm still searching for the appropriate model comparison for nested binomial GLMM. While I'm doing the search, comments from anyone knowledgable on this topic are greatly needed! $\endgroup$ – potpot_g May 20 '18 at 5:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.