How to deal with missing values of explanatory variables when comparing models

Question

I want to compare several logistic regression models. The different models are built using the same initial dataset. The models differ with respect to the explanatory variables included.

Many of the explanatory variables, however, have missing values. When estimating a model, I remove the observations for which one of the explanatory variables is NA. As a result, since observations will remain or be removed depending on the explanatory variables selected, the datasets used to estimate the different models are different. The datasets' size vary from around 150 observations to 500 observations depending on the combination of explanatory variables chosen.

Given the potential large difference in data size I do not solely compare models anymore which seems undesirable.

Would imputation here be advised such that I compare the models? Are there any other strategies I could follow?

Answer 1

I would use multiple imputation here, on the whole dataset, to produce several complete datasets. A rule of thumb is to use the average % of missing data as the number of imputed datasets to create - so if you have 30% missing data on average, then impute 30 complete datasets. Then run each of your models on all off the datasets, pool the results, and then use the usual methods for comparing models. Many packages that perform multiple imputation can handle the process including fitting the final analysis models and the pooling automatically once you specify the imputation model (that is, the way that the variables are related, and even this can be automated to a large extent).

If the data are missing completely at random (MCAR), or just missing at random (MAR) then a suitable imputation model can result in unbiased and more precise estimates. If you just delete missing cases then if the data are MAR you will likely produce biased as well as less precise estimates. If they are MCAR then you will produce less precise estimates.