How to implement Adaptive Lasso penalty for a Logistic regression in Python?

Question

I want to use an Adaptive Lasso instead of a standard Lasso because of the Oracle properties of the former. However, I cannot seem to find an option to implement an Adaptive Lasso for a logistic regression in Python. For example, the package asgl lets one implement penalties such as an Adaptive Lasso, however, it only supports linear and quantile regressions. Unfortunately, switching to R (glmnet e.g.) is no option here because of company regulation.

Does anyone know how to implement Adaptive Lasso penalties for logistic regressions in Python?

Answer 1

Adaptive LASSO is a two-step estimator; check out section 3.1 of Zou "The Adaptive Lasso and Its Oracle Properties" (2006). (This is the original paper that proposed adaptive LASSO.) You can implement the steps separately. Let $p$ be the number of regressors in your model.

1. You start with a $\sqrt{n}$-consistent estimator of $\beta=(\beta_1,\dots,\beta_p)^\top$ such as the MLE.*
2. For $j=1,\dots,p$, you specify $\tilde X_j$ as $\frac{X_j}{|\hat\beta_j|^\gamma}$ for some $\gamma>0$ (e.g. $\gamma=1$). You then run a standard LASSO using these modified $\tilde X$s instead of the original ones. (See Section 3.5.)

*This requires the number of regressors $p$ to be less than the sample size $n$, $p<n$. Otherwise, you need to look for another $\sqrt{n}$-consistent estimator.