1
$\begingroup$

glmnet is a widely-used R package for generalized linear regression. Among the arguments passed to the main function glmnet::glmnet(), the standardize controls whether the feature matrix is standardized before fitting, and the intercept determines whether an intercept is included in the linear model.

The thing is, glmnet allows the model to standardize the feature and fit with no intercept at the same time, which is confusing to me. Suppose that there is a feature $X$ standardized as $\tilde{X}=\frac{X-\mu}{\sigma}$, and $\tilde\beta$ is the estimated coefficient in front of $\tilde X$. When we recover the coefficient $\beta = \tilde\beta/\sigma$ for the unstandardized feature, a constant $-\tilde\beta\mu/\sigma$ is pulled out and added to the intercept. As a side result, an intercept is almost inevitable.

As the core algorithm of glmnet is written in Fortran, it is hard for me myself to look into its implementation.

$\endgroup$
0

1 Answer 1

1
$\begingroup$

The standardisation and intercept code isn't in the Fortran, it's in glmnet/R/glmnetFlex.R

Examining the code confirms that the intercept argument is about centering x and the standardize argument is about scaling x to unit variance, so they really are independent choices.

That is, standardize is about the units of the coefficients and intercept is about the choice of "zero" point for the sparsity.

if (intercept) {
        xm <- meansd$mean
    } else {
        xm <- rep(0.0, times = nvars)
    }
if (standardize) {
        xs <- meansd$sd
    } else {
        xs <- rep(1.0, times = nvars)
    }
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.