# Trade-off between omitting variables or dropping observations in multivariate logistic regression

Say you are selecting $$n$$ observations from a complex survey of $$N$$ individuals to create an analytical sample of relevant observations; and that you intend to fit a binomial multivariate logistic regression to the data, where:

$$Y = \beta_0 + \beta_1 D + \beta_2 X + \beta_3 Z + \epsilon$$

Say that for the covariate $$Z$$ included as control, there are $$K$$ missing observations. They are not missing completely at random and $$K > 0.1n$$.

How, mathematically, can you comparatively assess the sensitivity of your model coefficients and parameter to a choice of either dropping the K rows (listwise deletion) or omitting the Z feature?

(Related post: Missing data in a logistic regression analysis ; doesn't directly answer this question.)

Edit for clarification: Indeed you may be better off proceeding with multiple imputation; but here I am interested in understanding the consequences of the above choice in cases where a missing at random assumption is not necessarily plausible.

• You could fit the a model without the variable $Z$ and a model with $Z$ and omitted the $K$ rows, and check how the inference changes. Commented Apr 24, 2021 at 19:49

This is an unnecessary trade-off, as data don't have to be "missing completely at random" (MCAR) to include cases with missing values in the analysis. In practice, you often have data "missing at random" (MAR) in the technical sense defined in Stef van Buuren's Flexible Imputation of Missing Data (FIMD):

If the probability of being missing is the same only within groups defined by the observed data, then the data are missing at random (MAR). MAR is a much broader class than MCAR."

Put another way, MAR means missingness doesn't depend systematically on information not included in your data set. In that case, often a reasonable assumption, you use the information in the observed data to make multiple sets of estimates of the missing data (multiple imputations), fit your model to all of the imputed data sets, and then combine all the resulting models in a way that incorporates the uncertainty in both the imputations and the coefficient estimates.

In general, you don't want to omit any covariate associated with outcome from a logistic regression, as omission biases coefficients for included predictors downward in magnitude even if the omitted predictor is uncorrelated with those included. To estimate what would happen from listwise deletion of cases having missing values, you would need to have some model in mind for the missingness as a function of the cases and their covariate values. That essentially is an assumption of MAR, anyway. Section 1.3 of FIMD explains the dangers of listwise deletion and other solutions that seem "simple" at first but only work in limited circumstances. So why not explore multiple imputation and see where it leads?

In response to comment:

If you nevertheless want to compare listwise deletion of cases against covariate omission when MAR cannot be assumed, then you can draw from van Buuren's suggestion in Section 6.2 of FIMD when missing not at random (MNAR) is suspected:

perform a concise simulation study ... customized for the problem at hand with the goal of finding out how extreme the MNAR mechanism needs to be to influence the parameters of scientific interest.

You apply the same principles. Instead of performing and evaluating multiple imputation, however, you evaluate your alternative deletion strategies on sets of simulated data.

There is, however, no way to do this just "mathematically." As van Buuren notes later in discussing this type of sensitivity analysis:

In sensitivity analysis, imputations are generated according to one or more scenarios ... we should attempt to make an educated guess about both the direction and the magnitude of the missing data had they been observed. By definition, this guess needs to be based on external information beyond the data. (Emphasis added.)

The critical issue is not mathematical. The sensitivity analysis itself depends on your assumptions about why and how the data in your situation are missing beyond what you can glean from the observations themselves. That's particularly important for your logistic regression scenario with its inherent omitted-variable bias.

• Thank you for your answer. I understand your point and don't disagree. However, I am trying to understand better the implications of the abovementioned choice in cases where the missing at random assumption is not plausible, given that in my understanding this assumption is not testable in the data. In other words, I get that you generally don't want to do what I'm suggesting, but I'm trying to get a better understanding of the consequences of doing so. Commented Apr 24, 2021 at 15:43
• @atuin if MAR is not plausible then the data are missing not at random (MNAR); missingness then depends on information not present in your observations. If you don't know how missingness is associated with covariate values, outcomes, and the associations of covariates with outcomes, then there is no reliable way to know how either listwise deletion or covariate omission will work. It is possible to handle MNAR with imputation under additional assumptions and sensitivity testing of reasonable MNAR scenarios. See Section 6.1 of FIMD.
– EdM
Commented Apr 24, 2021 at 16:14
• @atuin I've added to the answer some suggestions for evaluating your alternative strategies of listwise versus covariate deletion, based on suggested ways for dealing with MNAR. The critical issue isn't mathematical: you must evaluate particular reasonable ways in which the data might be MNAR. Even for ordinary linear regression that has been done via simulation rather than theoretically; see the reference in Section 6.2 of FIMD. Again, if you're going to devote that much effort to the issue, going all the way to multiple imputation would make sense.
– EdM
Commented Apr 24, 2021 at 18:20
• Yes, that all makes a lot of sense, I've accepted the answer. I agree for the multiple imputation; but this was not for my own analyses, I was reading someone's paper and trying to understand better the consequences of their choices. Thank you! Commented Apr 25, 2021 at 4:09