# Penalized multinomial regression with missing values

I would like to fit some models on a dataset where I have a lot of missing values. I am especially interested in comparing models fit with and without imputed values, because the dataset has so many missing values (>50%) that it seems unlikely to me that any imputation procedure is going to yield sensible results.

I’m looking for a software package, ideally in R/Python, that could do all three of the following:

1. penalized regression (l1/l2/ElasticNet loss);
2. multinomial, linear, and logistic models;
3. handles missing values

In base R I can fit simple models using na.exclude. But neither scikit-learn nor glmnet can handle missing values. I know some methods, especially nearest-neighbor or decision tree-based methods, can naturally handle missing values, e.g. by adding another type of split or by omitting missing values in pairwise comparisons. Is there a package that can do all three of the above or is it just not possible?

You seem to ask how to use just one package in either Python or in R to do all the three tasks. This can be done in a "pipeline", in which you need to do some "pre-processing" to handle the missing values before you fit the model.

You can get the model fitting the training data in either Python or R.

In Python, you can use "sklearn":

from sklearn.preprocessing import Imputer
from sklearn.linear_model import LogisticRegression
imp = Imputer(missing_values='NaN', strategy='mean', axis=0)
mlg = LogisticRegression(multi_class='multinomial', penalty='l2')
model = Pipeline([('imputation', imp), ('multilogit', mlg)])


In R, you can use "caret":

library(caret)
model <- train(
x = X_var, y = y_var,
trControl = trainControl(method="cv", number=10, repeats=3, classProbs= TRUE,
summaryFunction = multiClassSummary)
method = "multinom",  # this specifies 'Penalized Multinomial Regression'
preProcess = "knnImpute"
)


As you say, xgboost (gradient boosted trees) is also robust to missing values. You can use the "xgboost" module/library in either Python or R to give it a try.

• Thanks for your comment, but I am specifically looking for an approach that can handle missing values without imputation - one of the issues here is that I am finding that in a univariate context, imputation significantly changes the set of analytes that are associated with my phenotype of interest. So I would like to handle the missing values naturally, if possible. – dentist_inedible Mar 14 '19 at 18:11
• I don't understand what you mean by "handle the missing values naturally." and you said you want to "handle missing values without imputation". If you don't want to delete the cases, the only option left here is to use xgboost, which doesn't do any imputation or data deletion. – Ray Yang Mar 14 '19 at 23:36