# How to implement Adaptive Lasso penalty for a Logistic regression in Python?

I want to use an Adaptive Lasso instead of a standard Lasso because of the Oracle properties of the former. However, I cannot seem to find an option to implement an Adaptive Lasso for a logistic regression in Python. For example, the package asgl lets one implement penalties such as an Adaptive Lasso, however, it only supports linear and quantile regressions. Unfortunately, switching to R (glmnet e.g.) is no option here because of company regulation.

Does anyone know how to implement Adaptive Lasso penalties for logistic regressions in Python?

Adaptive LASSO is a two-step estimator; check out section 3.1 of Zou "The Adaptive Lasso and Its Oracle Properties" (2006). (This is the original paper that proposed adaptive LASSO.) You can implement the steps separately. Let $$p$$ be the number of regressors in your model.
1. You start with a $$\sqrt{n}$$-consistent estimator of $$\beta=(\beta_1,\dots,\beta_p)^\top$$ such as the MLE.*
2. For $$j=1,\dots,p$$, you specify $$\tilde X_j$$ as $$\frac{X_j}{|\hat\beta_j|^\gamma}$$ for some $$\gamma>0$$ (e.g. $$\gamma=1$$). You then run a standard LASSO using these modified $$\tilde X$$s instead of the original ones. (See Section 3.5.)
*This requires the number of regressors $$p$$ to be less than the sample size $$n$$, $$p. Otherwise, you need to look for another $$\sqrt{n}$$-consistent estimator.
• (+1) So just to make things extra straightforward, in python this approach could be implemented by using LogisticRegression with the inverse regularization strength C=10000 or some other big number. Then, take each column of your data and divide it by $|\hat{\beta}_j|^\gamma$ where $\beta_j$ comes from the _coef field of the LogisticRegression object and $\gamma$ is a positive hyperparameter; perhaps $\gamma=1$. Finally, create a second LogisticRegression object using the transformed X variables and a more reasonable C (chosen in the usual manner(s)). Nov 10, 2022 at 15:10