If we want to conduct variable selection for a high-dimensional data with the binary responses, one good solution will be using L1 regularized logistic regression.

However, I wonder what will happen if I use L1 regularized linear regression (i.e. Lasso) for the binary response data. Particularly, I am hoping to find some statistical analysis of such a model misspecification. For example, given the oracle $\lambda$ (the regularization weight), what will the differences of estimating the effect sizes with these two methods.

I thought there are some papers discussing this, but I couldn't find any.


Erdogdu, Bayati, and Dicker have a nice paper describing this (without the L1 regularizer) in NIPs 2016. Their conclusion is that for large $p$ the differences should be minimal, and also provide a bound on the difference of the GLM vs the LM estimates. They do consider an L2 regularizer which may be of interest. My guess would be the complexity of the variable selection component might add something to the bounds (inequalities) that they state. Perhaps you could run some simulations equivalent to theirs and write up the results? I would be interested to see those results.

| cite | improve this answer | |
  • $\begingroup$ Big (+1)! This seems like the perfect paper to mention here $\endgroup$ – user795305 Sep 22 '17 at 1:32
  • $\begingroup$ Thanks! I will read this for inspiration for sure, but I think conclusions in $n>p$ case could be very different in the high-dimensional case where L1 regularization is necessary. (I agree that the differences between L1 and L2 might not be a problem, but whether $n>p$ or $n<p$ could be.) $\endgroup$ – Haohan Wang Sep 22 '17 at 2:27
  • 1
    $\begingroup$ If you are in a $p<n$ scenario then I'm not sure if the logic of their argument would still make sense, I know they are thinking about genetics applications where that might be true, if you email the author(s) they might be responsive to you enquiry about whether they have results-or simulations in the $p>n$ case. $\endgroup$ – Lucas Roberts Sep 22 '17 at 2:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.