Why doesn't glmnet handle missing data the way lm does?

Question

Computationally in R, I had some toy data, and I tried to fit an elastic net model using glmnet. I noticed that even if there was just a single missing value, the algorithm would not execute and it's advised to impute the missing value before hand.

# Set the seed for reproducibility
set.seed(123)

# Generate a 100x5 matrix of random numbers
data_matrix <- matrix(rnorm(100*5), nrow=100, ncol=5)
data_df <- as.data.frame(data_matrix)
data_df[1:1, 3] <- NA # Single missing value
# Generate a vector Y with 100 observations, each being 1, 2, or 3
Y <- sample(1:3, 100, replace=TRUE)
glmnet::cv.glmnet(x = as.matrix(data_df),
                           y = as.matrix(Y),
                           alpha = 0.5,
                           family = "multinomial")
Error in glmnet(x, y, weights = weights, offset = offset, lambda = lambda,  : 
  x has missing values; consider using makeX() to impute them

Algorithmically, what's causing elastic net not to fit when there's missing values? In contrast, when using lm and there's a missing value,

ctl <- c(4.17,5.58,5.18,6.11,4.50,4.61,5.17,4.53,5.33,NA)
trt <- c(4.81,4.17,4.41,3.59,5.87,3.83,6.03,4.89,4.32,4.69)
group <- gl(2, 10, 20, labels = c("Ctl","Trt"))
weight <- c(ctl, trt)
lm.D9 <- lm(weight ~ group)

The code ran without any errors.

Answer 1

The way that lm "handles" missing data is that it uses list-wise deletion -- the only cases that are retained & used to estimate the model are the ones for which all of the values in the formula are non-missing.

If you run the code for lm in OP's post, then summary(lm.D9) prints an output that includes this line:

Residual standard error: 0.716 on 17 degrees of freedom (1 observation deleted due to missingness)

And this line is buried in the middle of the output, so it's easy to overlook. And if we change the data to

ctl <- c(4.17, rep(NA,9))

then the message printed says

Residual standard error: 0.7937 on 9 degrees of freedom (9 observations deleted due to missingness)

In other words, even though nearly all of the data in ctrl is missing, there's only the faintest clue to that fact printed by summary.

Design

My opinion is that silently deleting data from the input is a bad design, because users may think that they're carrying out an analysis of all of the data that they give to lm, but in fact lm is only analyzing the complete cases (and silently deleting the rest).

Furthermore,glmnet's behavior -- raising an error when the data contains missing values -- is the preferred outcome, because it clearly communicates what happened and why. Even if the user intends to delete the cases with missing data, then there is a simple path to do so: filter the data first and pass only the complete data to glmnet. But the glmnet message is transparent about where the problem lies, and puts the user in the position of deciding how to solve it. By contrary, lm is guessing what the user is trying to do and making a “best effort.”