I am trying to fit a multinomial logit model using glmnet. I have a few questions:

  1. How is the baseline category specified?

  2. Looking at the model coefficients using coef.glmnet, I'm thinking that many are given by a dot. I assume this means the coefficient was set to zero by LASSO, so the variable was dropped. However, I'm finding that variables are dropped this way even when lambda=0, so there is no regularizing term. Can someone explain what's going on here?

  3. Sometimes I find that a variable is dropped in some but not all of the logits. How should this be interpreted?

Here is a working example:

X <- model.matrix(Species ~., data=iris)
y <- iris$Species
fit <- glmnet (X, y, family="multinomial")
coef(fit, s=0)
  • 1
    $\begingroup$ Please ask only one question per post ( you can link between them for common background) $\endgroup$ Sep 4 '18 at 19:59

Q1: glmnet do not use (or need) a baseline category. The penalization takes care of the linear dependence, no matrix is inverted during the fitting! Furthermore, it would be an error to use a baseline category, because different choices would lead to fitted models giving different predictions. See discussion in Dropping one of the columns when using one-hot encoding

Q2: Not sure about this, probably has to do with details about the algorithm (which is iterative, not exact). Leaving that one to others.

Q3: The usual lasso do not know about different dummys belonging to the same categorical variable, so it might choose to set different variables to zero. To avoid that, use the group lasso. See Why use group lasso instead of lasso?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.