Standard logistic regression post-Lasso The situation I'm interested in is logistic regression for a binary response variable with lots of predictors (500 to 1000), lots of which are correlated. I would like to use a logistic LASSO approach to identify the most promising predictors. I could either use that solution directly, or only use it to identify a concise set of good predictors and use standard logistic regression on only these predictors. 
I am aware of the analogous discussion regarding linear regression and LASSO(see, e.g. the recent paper by Belloni and Chernozhukov), but I have difficulties finding results that help me to make this decision in the logistic regression setting, and for practical purposes (prediction) rather than academical ones. I am also aware that logistic regression can also be viewed from an ordinary least squares perspective (see e.g. Hastie/Tibshiranis Elements of Statistical Learning, p. 124), but I currently don't see how to use this to transfer results from the linear to the logistic setting.
Could you give me pros and cons for both alternatives (direct logistic LASSO or standard logistic regression post-LASSO) or explain how the ordinary-least-squares-perspective on logistic regression can be used to transfer arguments from the linear regression case?
 A: This has been written about on the site in detail.  The lasso is meant to be a complete solution and it is completely inappropriate to use it to select features that are fed into a naive method that does not penalize for the context of having tortured data to find the features.  You are also making an implicit assumption that the lasso finds the "right" predictors.  Should you bootstrap the entire process you may be sorely disappointed to learn that in your case the features selected have a great deal of randomness in them.  This is more true in the case of co-linearities.  
If you want to emphasize parsimony over predictive accuracy, then model approximation (also called pre-conditioning) may be useful.  Here one uses the best available prediction method, e.g. penalized maximum likelihood estimation with a quadratic penalty, i.e., ridge logistic regression.  Then $X\hat{\beta}$ from that model is approximated using stepwise regression or recursive partitioning, etc.  The approximate model inherits the proper amount of shrinkage from the full model.  The approximate model may be chosen to yield $R^{2} = 0.95$ against the gold standard linear predictor.
