1
$\begingroup$

I want to use LASSO in R but shrinking towards some fixed vector $A$, instead of shrinking towards 0. The desired L1 constraint, given coefficient vector $\beta$, is $||\beta-A||_{1} \leq k$, rather than the standard LASSO constraint, $||\beta||_{1} \leq k$. I haven't been able to find any relevant articles.

For variables, $i$, selected by LASSO, $\beta(i)\ne A(i)$, and, for variables, $j$, NOT selected by LASSO, $\beta(j) = A(j)$.

For OLS, a solution is to define a shifted parameter vector, $\tilde{\beta}$, and shifted dependent variable, $\tilde{Y}$.

\begin{equation}\label{eq:Eq0} \tilde{\beta} = \beta - A , \end{equation}

\begin{equation}\label{eq:Eq1} Y = X\beta = X(A+\tilde{\beta}) = XA + X\tilde{\beta} , \end{equation}

\begin{equation}\label{eq:Eq2} Y - XA = X\tilde{\beta}, \end{equation} \begin{equation}\label{eq:Eq3} \tilde{Y}=X\tilde{\beta}. \end{equation}

This shows that LASSO using OLS run with $X$ and $\tilde{Y}$ would give the result $\tilde{\beta}$. The desired $\beta$ is found by adding $A$.

Unfortunately this doesn't help as my models are Poisson and logit. I cannot subtract $XA$ from a count or a Bernoulli rv and have anything sensible. An option would be to treat the $XA$ term in the equation as a constant. In R with lm and glm, the equation can defined with the offset() function which forces a coefficient of 1. This would be a solution but glmnet and other LASSO packages that I've encountered don't have this ability in the equation specification.

I wrote a LASSO implementation but it's slow and lacks important support such as cross validation. I'm hoping someone can point me towards a workable solution that doesn't necessarily have to be LASSO.

$\endgroup$
  • 2
    $\begingroup$ Use $A$ as an offset in your model. Search this site. $\endgroup$ – kjetil b halvorsen Mar 5 at 15:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.