Penalize the intercept in lasso (L1) penalized logistic regression or not?

Question

In logistic regression: $log(\frac{p(x)}{1-p(x)}) = \beta_0 + \beta_1x$,

let $x' = \frac{x-\bar{x}}{\sigma_x}$, then in terms of the centered and scaled varaible $x'$ , $$ log(\frac{p(x')}{1-p(x')}) = \beta_0' + \beta_1'x' = \beta_0' - \frac{\beta_1'}{\sigma_x}\bar{x} + \frac{\beta_1'}{\sigma_x}x $$ Since $log(\frac{p(x)}{1-p(x)}) = log(\frac{p(x')}{1-p(x')})$, then $$ \beta_1 = \frac{\beta_1'}{\sigma_x} \\ \beta_0 = \beta_0' - \frac{\beta_1'}{\sigma_x}\bar{x} = \beta_0' - \beta_1\bar{x} $$ (In logistic regression, intercept is the log odds only when other predictors are 0. )

Note the intercept $\beta_0'$ is not zero. But in the linear regression case, after centering and standardization, the intercept is 0, so there is no need to penalize the intercept.

Questions: in the logistic regression case, do we need to penalize the intercept ($\beta_0'$) after centering and scaling the data x?

Answer 1

In linear regression, if you center and scale x, the OLS (unregularized) intercept is the mean of y. In some cases, x and y are both centered and scaled; in that rescaled space, the intercept is 0 (which gives an unregularized intercept in the original space).

In logistic regression, if you center and scale x, you will still need an intercept. People generally do not penalize the intercept because they want full flexibility of the intercept. If you have prior belief that the intercept should be zero, then penalize it.