Group categorical variables in glmnet

Question

Consider the following fit:

fit3a=glmnet(x,g4,family="multinomial",type.multinomial="grouped")

How do I indicate which columns in x are categorical/multinomial? Is there an option to specify the index of the grouped variables?

The documentation describes the option type.multinomial as follows:

If "grouped" then a grouped lasso penalty is used on the multinomial coefficients for a variable. This ensures they are all in our out together. The default is "ungrouped".

Answer 1

The first two arguments that glmnet() is expecting are a matrix of the predictors (x, in your case) and a vector of the response (g4, in your case). For the x matrix, it is expecting that you have already dummied out any categorical variables. In other words, glmnet() does not actually know if any of your predictors are categorical, because they have already been dummied out.

If your data is in a data frame, a good way to construct the x matrix is using the model.matrix() function. It accepts formula language, will automatically exclude the response variable, and will create dummy variables for any predictors defined as factors.

The family="multinomial" and type.multinomial="grouped" options refer to the response variable having more than 2 possible outcomes. You can pass in the response variable (g4) as a factor.

The package authors provide a nice vignette explaining the usage of glmnet(), though it unfortunately does not give an example using model.matrix() to prepare the x matrix.

Answer 2

As justmarkham points out, you can construct the design matrix x using model.matrix. Note that you'll want to exclude the intercept, since glmnet includes one by default. You may also want to change the default contrast function, which by default leaves out one level of the each factor (treatment coding). But because of the lasso penalty, this is no longer necessary for identifiability, and in fact makes interpretation of the selected variables more complicated. To do this, set

contr.Dummy <- function(contrasts, ...){
   conT <- contr.treatment(contrasts=FALSE, ...)
   conT
}
options(contrasts=c(ordered='contr.Dummy', unordered='contr.Dummy'))

Now, whatever levels of a factor are selected, you can think of it as suggesting that these specific levels matter, versus all the omitted levels. In machine learning, I have seen this coding referred to as one-hot encoding.

Assuming that g4 has K levels, the type.multinomial="grouped" option specifies that the features of x will all enter the model simultaneously for each of the K linear predictors, as opposed to having the linear predictor for each class (in general) having its own features. glmnet does not (currently?) support grouped-type penalties of predictors (the x matrix). The package grplasso does, but is written in pure R, so is slower than glmnet, but you could give that a try.