# How to interpret elastic net coefficients for multinomial regression?

I ran a elastic net regression path for a multinomial model using the glmnet package in R. My response variable has 3 levels (0, 1 and 2 for 3 different stages of a disease).

Here are the coefficient output from the elastic net

# Stage0
# (Intercept) -0.14301343
# Age          -0.16305154
# Q1       -0.04925107
# Q2       0.01030530
# Q5      0.11559569
# Q9      0.01927071

#Stage1
# (Intercept) -0.9571839
# Q5           0.4063393

#Stage2
# (Intercept)  1.10019732
# Q2      -0.04281563
# Q4      -0.07533350
# Q6      -0.01961934
# Q7      -0.15523562
# Q9      -0.03297465
# Q17     -0.06386999


My predictor variables are age, and Q1-Q18 for 18 items on a psychiatric questionnaire. I see that the elastic net shrunk certain coefficients to 0. My questions are

1) In the output above, the coefficients are separated by my response level (stage0, stage1, or stage2). What exactly does this mean? Is it that the coefficients under Stage0 are for the model when the multinomial model uses Stage0 as the reference level? How do I interpret these coefficients?

2) Can I take these remaining variables and fit a multinomial regression model? So instead of using all of my original 19 predictor variables, I can just focus on those that were selected by elastic net and do model building from there?

• 1) I recommend asking the first question as its own question, as interpreting the coefficients of a multinomial logistic regression is different from how to deal with coefficients set to zero by LASSO. // 2) Your $y$ sounds ordinal. In such a case, ordinal models might make more sense than multinomial.
– Dave
Commented May 6, 2022 at 18:41

(I will address the second question.)

NO

While you no longer have to collect data on the variables who coefficients are estimated as zero, eliminating those variables and refitting will (or at least can) change the other coefficient estimates. During the minimization of the LASSO loss function, the features that wind up "dead" still contribute by setting their coefficients to be zero.

I find it plausible that the math could work out to give that the penalized coefficient estimates happen to be the same as the unpenalized coefficient estimates for only the surviving features. That would be an elegant result, but it happens not to be correct.

• (+1) for what it's worth, the point isn't that the "dead" coefficients still contribute. Elastic net both shrinks and selects, by design. The answer to whether the coefficients of a relaxed elastic net are the same as the coefs from the elastic net is "no" because of the shrinkage.
– Ben
Commented May 6, 2022 at 18:50