Selecting the best subset of variables for parsimonious binary logistic regression models In addition to PROC VARCLUS, randomForest, glmnet, and assessing multicollinearity among potential predictor variables (without regards to the outcome of interest), I am seeking other methods of variable selection in lieu of using stepwise methods for building more parsimonious binary logistic regression models (containing 8 to 12 variables to predict outcomes such as loan payment/default or current/late payment history) from a wide array of potential predictor variables (500+ variables, 200k+ records). 
Below I have included an R script using FSelector to select the 8 highest "ranked" variables:
library(FSelector)
fit <- information.gain(outcome ~ ., dataset)
fit2 <- cutoff.k(fit,8)
reducedmodel <- as.simple.formula(fit2,"outcome")
print(reducedmodel)

I have two questions regarding this script and the FSelector algorithm in general:   


*

*Is the information.gain criteria in the above script synonymous with Kullback-Leibler divergence?
If so, can someone explain this in more layman terms than Wikipedia as I am relatively new to this area of statistics and would like to start off with the right idea of this concept as I may likely use this approach a great deal in the future?

*Is this a valid approach, if there is such a thing as a valid approach, to select a desired number of variables for a binary logistic regression model (e.g., selecting the 8 highest "ranked" variables for use in a parsimonious model)? If not, can you provide an alternative approach to do so?
Any insight or references regarding this topic and/or these questions will be greatly appreciated!
 A: Variable selection without penalization is invalid.
A: Rarely does a question appear that I believe I can answer.
This isn't in R (at least I don't believe there is a addin for it).  It doesn't answer your exact questions either, although I believe may be of help.

I am seeking other methods of variable selection in lieu of using
  stepwise methods for building more parsimonious binary logistic
  regression models (containing 8 to 12 variables to predict outcomes
  such as loan payment/default or current/late payment history) from a
  wide array of potential predictor variables (500+ variables, 200k+
  records).

Perhaps Correlated Component Analysis would answer your problems.
See here: http://www.xlstat.com/en/products-solutions/ccr.html

Predictor Selection Using the CCR/Step-Down Algorithm 
  In step 1 of the step-down option, a model containing all predictors is estimated
  with K* components (where K* is specified by the user or determined by
  the program if the Automatic option is activated), and the relevant CV
  statistics are computed. In step 2, the model is then re-estimated
  after excluding the predictor whose standardized coefficient is
  smallest in absolute value, and CV statistics are computed again. Note
  that both steps 1 and 2 are performed within each subsample formed by
  eliminating one of the folds. This process continues until the
  user-specified minimum number of predictors remains in the model (by
  default, Pmin = 1). The number of predictors included in the reported
  model, P*, is the one with the best CV statistic.

Here is the summarization for when it was presented at the Paris workshop on PLS developments 2 years ago.
Jay Magidson. "Correlated Component Regression A Sparse Alternative to PLS Regression." http://statisticalinnovations.com/technicalsupport/ParisWorkshop.pdf
It sounds like what you are partially trying to accomplish is done with this algorithm in this tutorial: 
"Using Correlated Component Regression with a Dichotomous Y and Many Correlated Predictors."
http://statisticalinnovations.com/products/xlstattutorials/XLCCRtutorial2.pdf
I also hate it when people answer questions by just referencing other works instead of actually answering the question, but, I just feel this is the best answer if I'm understanding everything correctly.  Hope it helps.
A: To answer the above two questions:
1) In layman terms, Kullback-Leibler divergence, as displayed in R when using the FSelector package, is the relative amount of information that can be gained by using a given potential predictor variable.
2) It is NOT a valid approach to select a desired number of variables for a binary logistic regression model (e.g., selecting the 8 highest "ranked" variables for use in a parsimonious model) as no such approach exists. Most importantly, it does not take into account collinearity among predictor variables, as it simply rank orders variables by information gain. Information gain is very similar to the c-statistic or information value in terms of being a univariate measurement for classification strength for a given predictor variable. As such, it is useful to rank variables, but other methods must also be applied to further build parsimonious models (eg. PCA and Lasso or Ridge Regression).
I hope this helps someone!
