4
$\begingroup$

Are there any easy to use alternatives to stepwise variable selection for GLMMs? I have seen implementations of e.g. LASSO for linear regression, but so far not seen anything for mixed models. Mixed models seem non-trivial in general, so I am wondering if any of the fancy new methods have been adapted from them (and possibly implemented in R). Using whatever selection procedure you like and then validating the results seems a sensible way to go in the meantime.

To give some context: in my current project, I am looking at approximately 700 variables and 5000 binary observations. Stepwise selection takes about 1 day; many variables have about 10% missingness.

Edit: Thank you for the very interesting answers so far! Two concerns that I have are: do these new methods have longer runtimes than stepwise selection and can they deal with missing data (if each variable has different missingness, than for hundreds of variables it is very easy to loose all observations in a complete case analysis - something that stepwise selection can deal with by only using small subsets of the available variables at the same time).

Improve this question
$\endgroup$
  • 2
    $\begingroup$ Look a bit closer: glmmLasso exists. $\endgroup$ – usεr11852 Sep 30 '15 at 23:51
  • $\begingroup$ Thank you @usεr11852 . It is great to know this method exists. It seems to be very fast (compared to stepwise selection). However, unlike stepwise selection it limits us to a complete case analysis, which can be a problem when using a large number of variables with different missingness. I did get a "Fisher matrix not invertible" error, but my design matrix might be rank-deficient. If you do have suggestions to avoid a complete case analysis (by only looking at small subsets of the variables at the same time), please let me know. Also, please feel free to convert this to an answer. $\endgroup$ – Rob Hall Oct 1 '15 at 18:47
  • 1
    $\begingroup$ To solve the missing data problem, consider multiple imputation. This paper provides a way to combine multiple imputation with LASSO; on first glance it seems that it should work with glmmLasso as well but may take some effort to implement. Alternatively, see if you can combine your 700 variables into a smaller set based on subject-matter knowledge, to minimize the missing-data problem. $\endgroup$ – EdM Oct 1 '15 at 19:09
  • $\begingroup$ @EdM thank you for the very interesting article. I have so far avoided MI as my particular goal is prediction rather than inference (and stakeholders have some concerns about a large amount of imputation). At first glance, the paper seems to involve imputing in the usual fashion, then doing LASSO in the usual fashion and then aggregating results in a novel fashion, so it might be compatible with existing implementations. This looks like it might enable me to solve my problem if I can get it all to work. $\endgroup$ – Rob Hall Oct 1 '15 at 19:17
1
$\begingroup$

How about the ensemble method of boostrapped aggregating, also known as bragging? Using this approach you essentially create a large number of replicates of the original dataset using simple random sampling with replacement (say 10,000 bootstrapped datasets) from your original dataset. Then you implement a variable selection routine (perhaps best subsets or traditional stepwise selection methods) to select the coefficients or predictors that are significant for each of the boostrapped samples. You perform the routines for each bootstrapped samples and then look at the rates of how often the predictors are selected. Predictors that appear in say 90% or more of the sample are then used in the final mixed model. There are many other methods that could be used too, but I highlight this one as it's simple to explain and usually very easy to implement. For more information see, Breiman, Leo (1996). "Bagging predictors". Machine Learning 24 (2): 123–140. doi:10.1007/BF00058655.

Improve this answer
$\endgroup$
  • 1
    $\begingroup$ (1) do you mean bagging and not bragging in the first line? (2) does it make sense to do simple random sampling when the data are grouped (since it's a mixed model)? $\endgroup$ – Ben Bolker Oct 1 '15 at 2:53
  • $\begingroup$ Hi, Ben. Yes, I mixed a little terminology. In theory one could use bagging or bragging (bagging uses the mean and bragging using the median -- the "r" in "bragging" standing for "robust" indicating use of the median. To be fair "Braggin" with an "r" was introduced by Buhlman in 2003, but is an extension of Breiman's work. You make a good point regarding the bootstrap sampling scheme. If the data are grouped you would want to select the cluster's randomly rather than the individual observations or uses a sampling method that is consistent with the how the data were collected. $\endgroup$ – StatsStudent Oct 1 '15 at 3:14
  • $\begingroup$ Thank you @StatsStudent. This definitely sounds like an interesting approach. If I understood it correctly, it will involve repeating the variable selection quite a large number of times (I suspect similar to traditional bootstrap 1,000 might be a lower limit). This may limit its applicability, as one run of stepwise selection often takes me a day to do. $\endgroup$ – Rob Hall Oct 1 '15 at 18:20
  • $\begingroup$ Yes. That's correct, Rob. If it's taking you a day to do it, I'd suggest using a smaller subsample of your larger datasets and perform the repeated stepwise regressions on the smaller, more computational feasible datasets. You might also want to look into Graphical Processing Unit (GPU) processing to speed the computation time with massive datasets. But I find sampling usually does the trick for me. Take a smaller, but sufficiently large bootstrap samples, and then then compute on this. Take the coefficients that appear most of the time & build your final model with all the data on it. $\endgroup$ – StatsStudent Oct 1 '15 at 23:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.