What is the algorithm to solve logistic regression with lasso a.k.a $L_1$ regularization?

Question

what is the procedure/algorithm for a logistic regression with a penalty on the $L_1$ norm?

I found the iterative soft thresholding algorithm for a classical linear regression, but I can't adapt it for a logistic regression.

Answer 1

Default logistic regression minimizes below function:

$L(f(X, \beta), Y) = \frac{1}{N} \Sigma_i^N \left[-y_i log(f(x_i, \beta)) + (1-y_i)log(1-f(x_i, \beta))\right]$

Logistic regression with L1 penalty minimizes below function:

$L(f(X, \beta), Y) = \frac{1}{N} \Sigma_i^N \left[-y_i log(f(x_i, \beta)) + (1-y_i)log(1-f(x_i, \beta))\right] + \lambda \Sigma_i^K |\beta_i|$

Where:

$f(x_i, \beta) = \frac{1}{1+e^{-\beta^T x_i}}$

and $K$ refers to the dimension of $x_i$

Difference is second bit, that is:

$\lambda \Sigma_i^K |\beta_i|$

If you are using Python, it is already implemented in sklearn. For any other language which L1 Logistic regression is not implemented, you can code up the above function and use any iterative optimization algo to obtain $\beta$, such as gradient descent.