Alternatives to stepwise logistic regression with LARGE datasets After reviewing related questions on Cross Validated and countless articles and discussions regarding the inappropriate use of stepwise regression for variable selection, I am still unable to find the answers that I am looking for in regards to building parsimonious, binary logistic regression models from datasets with 1000 (or more) potential predictor variables.
For some background information, I typically work with large datasets, 500k or more rows, and my interest is in building binary logistic regression models to predict whether an individual will pay (1) or not pay (0) their bill on a particular account without using stepwise logistic regression. Currently, stepwise logistic regression is hailed as the “perfect method” among other statisticians that I have worked with, and I would like to change that as I have witnessed many of its pitfalls firsthand. 
I have recently dabbled in PCA (proc varclus) and random forest analyses (randomForest) with the latter being especially helpful; however, I am still seeking further direction on how to reduce the number of variables in my binary logistic models without using stepwise logistic regression. With that being said, any help (suggested articles or thoughts) is greatly appreciated. Thanks!
 A: Another option is the lasso, which can efficiently select a subset of important predictors from large collection.  This can be implemented in the glmnet package in R.  See http://www-stat.stanford.edu/~tibs/lasso.html for more information.
A: Tree-based methods (e.g., CART) are widely used for solving this problem.  While I would always prefer to estimate a logit model using theory, when in situations where there is insufficient theory or the data is not well understood, a tree-based method is, in my experience, always preferable to a logit model as it scales better to large data sets, is much more robust and better deals with non-linearities and interactions.  If you superiors are desperate for a logit model, you can use the variables selected by the tree-based model in a logit model.  
A: A couple ideas: Rather than PCA, you might look into partial least squares, which tries to account for relationships both among the IVs and between the IV and DV. I've mostly used it for p >> n problems, but it could be helpful in your situation, too.
With so many variables, surely many of them are highly correlated? Since we are clearly dealing with people - I don't think you could come up with that many IVs and not have high correlations. You could check for all correlations over, say, .9, and randomly delete one of each such pair. Admittedly, an ad hoc procedure, but I've found it useful.
