The situation I'm interested in is logistic regression for a binary response variable with lots of predictors (500 to 1000), lots of which are correlated. I would like to use a logistic LASSO approach to identify the most promising predictors. I could either use that solution directly, or only use it to identify a concise set of good predictors and use standard logistic regression on only these predictors.
I am aware of the analogous discussion regarding linear regression and LASSO(see, e.g. the recent paper by Belloni and Chernozhukov), but I have difficulties finding results that help me to make this decision in the logistic regression setting, and for practical purposes (prediction) rather than academical ones. I am also aware that logistic regression can also be viewed from an ordinary least squares perspective (see e.g. Hastie/Tibshiranis Elements of Statistical Learning, p. 124), but I currently don't see how to use this to transfer results from the linear to the logistic setting.
Could you give me pros and cons for both alternatives (direct logistic LASSO or standard logistic regression post-LASSO) or explain how the ordinary-least-squares-perspective on logistic regression can be used to transfer arguments from the linear regression case?