How to conduct predictor selection in a generalized linear mixed model? I have 18 predictors in a binary generalized linear mixed model (repeated measurements, over a 1000 subjects). I would like to trim the model a bit and remove some noise and useless predictors. Unfortunately, PROC GLIMMIX does not have any facility to do this. I could not find an R package that would do this (step() function style). If I were to try this manually, say begin with a full model (all predictors in) and do a 'backward selection', what criterion could I use to do this quickly? Could I use, say, p-values? But at what significance level?
 A: Rather than using a stepwise procedure, I would fit an L1-regularized model, and discard predictors whose coefficients are effectively forced to be zero. See [Ng 2004].
A: The most robust way to do backward selection using a given model, would be to remove one predictor, fit the model to the the remaining predictors and evaluate (in cross-validation of course). Repeat for all predictors and remove the one which has the smallest contribution. Then iterate the process until you achieve a combination of predictor number/performance that satisfies you. If you have 18 predictors, it would take 18+17+16+15+..+(k-1) model fittings/evaluations to find a model with k predictors.
If this your model fitting/evaluation is too expensive in term of resources, you can try any kind of predictor evaluation instead, you can use one of several possible measures (e.g. statistical tests/correlation/information gain) and use the same process of fitting and evaluating. You don't care about a threshold because at each iteration you remove the one with the lowest contribution regardless of the actual value. The true evaluation will be on what you are trying to predict (CV etc.).
