1
$\begingroup$

When you penalize logistic regression using l1, l2 or both penalizations, the coefficients are penalized towards 0. I would like to do the same thing but penalizing the coefficients towards other different numbers. For instance, penalize the first coefficient towards 1, the second towards 10 and the third towards -3.

Note that I am aware of this question and its answer, but it wouldn't quite work in a logistic regression scenario as substracting some combination of the predictors to the target variable doesn't give a binary variable, therefore the transformed regression is not a logistic regression anymore.

If somebody finds an R package or a way to do this with glmnet I would appreciate it a lot, thanks.

$\endgroup$
1

1 Answer 1

3
$\begingroup$

This will depend on the software used, and with R possibly with the package. But the concept that you need is that of an offset. An offset is a variable which you include in a linear predictor with a known, given constant coefficient. In R modeling functions which accepts a formula argument, it can be given as y ~ offset(x) which uses x in the linear predictor with an offset of 1. So, if you want that offset to be 3, use y ~ offset(3*x). If you want regularization against 3, you would need y ~ x + offset(3*x).

But glmnet do not accept a formula argument, but it has an extra optional argument offset which can be used likewise. So read ?glmnet. But there is a contributed package glmnetUtils which offers a formula argument, so I would use that package.

$\endgroup$
3
  • $\begingroup$ Ok, I get it, and I guess that y ~ x + offset(3*x) doesn't imply any collinearity problems among the predictors, as it deals with offsets in a different way? $\endgroup$ Jan 4, 2019 at 12:19
  • $\begingroup$ No collinearity problems, as it is just an additional constant in the linear predictor. $\endgroup$ Jan 4, 2019 at 12:25
  • $\begingroup$ Are you sure this is the best way? Can one not replace the weighted least square step in the IRLS algorithm (cf algo here stats.stackexchange.com/questions/178492/…) by a nonzero-centered ridge regression (see sciencedirect.com/science/article/abs/pii/S0047259X21001603 and refs therein), which can be obtained by row-augmenting the covariate matrix sqrt(W)*X with a diagonal matrix with sqrt(lambdas) along the diagonal and adding a vector center * sqrt(lambdas) to the outcome variable z * sqrt(W) in each iteration? $\endgroup$ Jun 11, 2023 at 7:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.