# A data set with missing values in multiple variables

I'm trying to analyze a set of data related to the health area but I'm not sure how to proceed with the missing values.

Objective: To adjust a model with a discrete response, to study the influence of certain variables on the response.

The scenario is as follows

• 20 variables
• A variable with 95% missing data
• A variable with 53% missing data
• A variable with 52% missing data
• Two variables with 2% missing data

Initially I thought about discarding this variable with many missing values, however it seems to me important for my analysis. The second option would be to work only with observations with complete data, but I do not know if this could bring problems for my analysis.

I have no knowledge about imputation, is there any way to proceed in this case?

EDIT: Is viable in this case with this number of variables use a multivariate technique like principal components?

EDIT2: My data set is basically a medical record of every patient who was admitted to a hospital. The missing data are not random, simply not all information was collected for all patients

• My rule of thumb is discard any variable that has more than 50% missing values. I would be very careful about imputing missing values, especially in the health related data. This review might be useful personal.psu.edu/jxb14/M554/articles/Graham2009.pdf – discipulus Mar 9 '17 at 0:04
• 50% seems pretty arbitrary. If the variable is potentially very important discarding it essentially means discarding the whole analysis / investigation... – Björn Mar 9 '17 at 19:26

@Tim gave a nice response. To add to that, the best thinking about dealing with missing values (MVs) began with Donald Rubin and Roderick Little in their book Statistical Analysis with Missing Data, now in its 9th edition. They originated the classifications into MAR, MCAR, etc. To their several books I would add Paul Allison's highly readable Sage book Missing Data, which remains one of the best, most accessible treatments on this topic in the literature.

A number of commonly used, bad heuristics have emerged over the years for dealing with missing data, many of which still see use today since they are easily implemented "solutions." These include ones already mentioned such as discretizing the variable and creating a junk category labelled "Missing" or "NA" (not available, unknown) into which all missing values for that variable are tossed, as well as, for continuous variables, plugging the missing values with a constant -- e.g., the arithmetic mean. Secondarily and for regression models, some recommend using dummy variables (0,1) indicating the presence (absence) of an MV. The dummy is intended to "capture" the overall impact of the MVs on the model while also appropriately adjusting the parameters. These are all bad ideas because, in the first case, a heterogenous mix of information is lumped into a single category while, in the second case, a potentially large burst or spike containing a single value (the mean) is introduced into an otherwise typically smooth distribution for a predictor.

The least biasing of all of the options for imputation are regression models. In an American Statistican paper (for which I no longer have a reference, sorry), dummy variables for MVs in regression have been demonstrated to not only not capture the effects of missing values but also to generate biased parameters. The AmStat paper based these conclusions on a comparison of the scenarios for the various MV options with full information data. The author's recommendation was, assuming the magnitude or volume of missing information wasn't too much or too large, to use the least biasing solution -- full information modeled imputation based on data available after deleting the observations containing MVs. Of course, this response demands an answer to "what is too much?" Here, there are no firm benchmarks, only experiential, subjective heuristics and rules of thumb without any firm theoretical motivation. This means that it's up to the analyst to decide. Just so, @Discipulus' rule of thumb is to work with variables containing 50% or less MVs, certainly a reasonable heuristic. In the OPs case, that would exclude the two variables containing more than 50% MVs, variables that are described as "important" to the analysis. That said, it is safe to assume that 95% MVs qualifies as "too much."

If it's thought that there are not too many MVs, then use some variant of multiple imputation to plug them. Here too, there are many bad methods to choose from including, e.g., "sorted hot deck" multiple imputation where observations are sorted across a string of fully observed variables and the fully observed value that comes closest to the observation with missing information across that sort string is used as a plug. In general, all of these "mechanistic" solutions to plugging MVs are to be rejected in deference to model based multiple imputation.

In an ASA workshop taught by Rubin, several "best" practices were discussed for dealing with multiple variables containing MVs in a dataset. First, rank the variables by their frequency or percent of missing information from high to low and begin the process of imputation, one variable at a time, on those containing the lightest or least amount of MVs. Then, retain and use these newly plugged variables in the model-building process for each subsequent variable. Use every variable available to you in building imputation models, including the target or dependent variable(s) and excluding the lower ranked variables with MVs.

The key metric in building and evaluating model-based imputation is the comparison of the pre-imputed means and std devs (based on the full information after deleting MVs) with the post-imputation or plugged value means and std devs. If the imputation was successful, then little or no (significant) difference should be observed in these marginals. An important note of caution needs to be introduced at this point: this metric and multiple imputation in general is intended to evaluate the preservation of overall or unconditional marginals. This means that the actual values used and assigned to each MV field have a high likelihood of being "wrong" for that observation, if compared with the full but unavailable information. For instance, in a head-to-head comparison of actual vs imputed values based on a sample possessing both self-reported survey information (the actual values) versus the imputations made by a leading vendor of geo-demographic information (the imputed values), imputed fields such as head of household age and income were wrong nearly 80% of the time at the level of the individual observation. Even after partitioning these fields into high and low groups based on median splits, the imputations were still wrong more than 50% of the time. However, the marginals were recovered more or less accurately.

One final note, imputation can be appropriate for features, predictors or independent variables but is not recommended for the target or dependent variable.

It all depends on why the data is missing. If the data is Missing Completely At Random, you can discard the incomplete data. If the data is Missing At Random, your best bet is multiple imputation (e.g., check out the mice or mi packages in R, and the various blog posts that describe how to use multiple imputation).

However, with the type of data you have, it is typical that the missing data will not be "randomly" missing in any sense of the word. For example, if data is missing because people did not get specific procedures, or were not given specific tests, or only stayed in hospital for a short time so little data was collected, imputation is a poor way to go, as your missing data is nonignorable. A solution is to treat each of your predictors as being categorical, where one of the categories is called "No data". As to whether this is the best solution, there is no way to say without a whole lot more detail on your data, problem, and the model.

• Your second paragraph helped me a lot to think about the problem. My data set is basically a medical record of every patient who was admitted to a hospital. The missing data are not random, simply not all information was collected for all patients. – user72621 Mar 9 '17 at 2:50

Before you decide on whether you want to impute, you should ask yourself whether the patients not giving you information should be part of your model. For instance, if you want to model for control vs treated, all those missing patients should be dropped (they don't tell you which group they are in). However, if you want to construct just a contingency table (or just exploratory analysis) of all patients in the hospital, you may want to keep the patients.

You will need to have good reasons for imputation because your data is potentially sensitive. For instance, if you impute a patient with the average cancer rate, what if the patient doesn't have cancer? How are you going to explain your boss that you guess a model for predicting cancers?

Any imprecision could be potentially deadly in your data set. Since your missing data is not random, the most common strategy is to recode your variables such that you have a new "NA" category.