Longtime lurker here.

I have a question about determining informative variables in generalized linear mixed models (GLMMs). My background is ecology, and I primarily examine habitat selection under a case-control (or "used-available") framework, typically using a binomial distribution.

Given that conventional GLMs are much easier and faster to fit than GLMMs, why is it not advisable to first determine the optimal fixed-effects structure in a fixed-effects only (conventional GLM) framework, and then determine the appropriate random effects?

My assumption here is that there would never be an informative parameter in a GLMM that is not also informative in an equivalently specified GLM, but not vice versa. Is this not the case?

I have read through Zuur et al. 2009, but they focus more on LMMs and recommend the opposite strategy (determine RE structure first using REML, then FE structure using ML). I've also read here that REML is perhaps poorly defined for GLMMs, so I don't know that their strategy is valid.

Thank you!

  • $\begingroup$ Maybe I am missing something but it is because the assumption of independence of errors is violated. $\endgroup$ – Peter Flom Jun 20 '19 at 9:51

A couple of points:

  • The reason why you typically need to use GLMMs, i.e., include random effects, is to account for correlations you have in your outcome data in specifics clusters/groups. For example, measurements taken on the same habitat will be correlated.
  • To select the fixed-effects structure you need first to appropriately model these correlations. Depending on how strong these correlations are, if you will use statistical tests based on a GLM that ignore these correlations, you will get invalid p-values.
  • $\begingroup$ Right, I understand why we use random effects, and that p-values obtained by the GLM would be invalid, but wouldn't they tend towards type 1 error only? e.g. GLM based p-values indicating a variable is significant when in fact it may not be under a GLMM. So my thought would be to do the initial round of variable selection under GLM, and then switch to GLMM, simply because the GLM is easier to fit with a lot of variables. $\endgroup$ – Tony K Jun 20 '19 at 20:59
  • $\begingroup$ Depending on the type of effects you’re looking at (i.e., within clusters or between clusters effects) the p-values can be wrongly too small or wrongly too large. Hence, doing the selection based on a GLM can go wrong. $\endgroup$ – Dimitris Rizopoulos Jun 21 '19 at 4:02
  • $\begingroup$ Ah, ok. I had compared the results from GLMs and GLMMs on my data, and the p-values were consistently too small for GLMs, but I wanted to be sure that was universal and not just particular to that dataset. Sounds like the p-values are unpredictably wrong. Thank you for the answer! $\endgroup$ – Tony K Jun 21 '19 at 18:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.