scikit learn: add lasso or ridge penalty only on subset of parameters

Question

Is there a way of using the linear model api to add the lasso penalty for a subset of the parameters I am regressing? For example, consider a linear separable decomposition that I want to fit to some sparse data subject to smooth constraints in one of the (separable) dimension.

This question indicates that this is somewhat possible through R's glm.

Is there LASSO type model in which only some of the regressors are regulated?

Of course, a DIY method will not be too hard. The question is rather whether it is possible through the scikit learn api.

Answer 1

I wrote a module for this purpose. You can take a look:

https://github.com/3zhang/Python-Lasso-ElasticNet-Ridge-Regression-with-Customized-Penalties

Answer 2

It is possible for ridge regression but not lasso as of scikit-learn 1.3.0. Compare the docs on the penalty weight alpha in Ridge, which does allow you to set an array of weights, vs LASSO, which does not.

I goofed. Setting alpha as an array is for performing multiple regressions with the same predictors against different targets. To do a ridge regression with different penalties on each coordinate of $x$, note that $$ \|Ax-b\|^2 + \alpha \|x\|^2 = \left\|\left[\begin{matrix}A\\ \alpha I\end{matrix}\right]x - \left[\begin{matrix}b\\ 0\end{matrix}\right]\right\|^2. $$

So to only apply alpha to certain coordinates you have to run a regular linear regression, but append the relevant rows of $\alpha I$ to $A$ and an equal amount of zeros to $b$.