# Missing data and imputation in general

Handling missing data is a bit confusing for me. My questions are:

1. Is it better to calculate imputations than simply leave out NAs and leave it to the (appropriate) model to handle it?

2. Is there a common threshold in the column/whole dataset under which imputation generally not executable? I suppose, if there are only 3 observations in a 100,000-row table, it is nonsense to calculate the missing ones...

3. Is it better to leave categoricals as such or should one convert it into dummies for better efficiency?

4. I have learnt that R imputation packages (e. g. Amelia, mice) produce several "imputations" for NAs, but none of them combine them into a single "most probable" set. What can one do with m different imputations?

The following are my thoughts on the subject (as per your questions):

Is it better to calculate imputations than simply leave out NAs and leave it to the (appropriate) model to handle it?

I think that the answer to this is: it depends. Some models (or, more accurately, software that implements those models) can handle missing data automatically, due to implemented algorithms of either handling missing data per se, or embedding multiple imputation or similar methods into the modeling software (usually, functions, i.e. in R). Therefore, you need to carefully read the software's documentation to see what missing data handling features it offers to the user.

Another important point in finding the correct or optimal answer is determining (testing assumptions about) the nature/mode of missingness. I'm talking about MCAR, MAR, MNAR - for more details on this and, in general, for a comprehensive overview of the topic as well as approaches, methods and software for missing data handling, see the excellent paper by Horton and Kleinman (2007).

Is there a common threshold in the column/whole dataset under which imputation generally not executable? I suppose, if there are only 3 observations in a 100,000-row table, it is nonsense to calculate the missing ones...

I have not seen any common thresholds for this. Your example above is an extreme case and does not represent most of real data sets. Moreover, even a small level of missingness (say, several percentage points) in many variables might produce significant overall missingness in the model: "... missingness of just a few percent on each of a number of covariates may lead to a large number of observations with some missing information" (Horton & Kleinman, 2007, p. 79).

Is it better to leave categoricals as such or should one convert it into dummies for better efficiency?

As far as I know, in most cases it is OK to use categorical variables as is, of course, assuming that the software you're using supports that. Most software indeed has direct support of categorical variables - see paper by Horton and Kleinman (2007) for details. Perhaps, there exist some situations, when it would be beneficial to convert them, but as of now I'm not aware of such.

I have learnt that R imputation packages (e. g. Amelia, mice) produce several "imputations" for NAs, but none of them combine them into a single "most probable" set. What can one do with m different imputations?

To the best of my knowledge, this is not true. Both Amelia and mice provide functionality for aggregating the imputed results and even performing some types of statistical analysis. Even more integrated process can be found in the R-based Zelig software, which supports various statistical models and has an embedded support for missing data handling (via Amelia package).

NOTE: Keep in mind that Amelia, in addition to traditional MAR assumption, also has an assumption that the data you're trying to process is multivariate normal. So, if it is not the case, other options should be considered, such as mice or corresponding Hmisc functionality.

References

Horton, N. J., & Kleinman, K. P. (2007). Much ado about nothing: A comparison of missing data methods and software to fit incomplete data regression models. The American Statistician, 61(1), 79–90. doi:10.1198/000313007X172556 Retrieved from http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1839993

It depends on who you ask, what you are trying to accomplish, the pattern of missingess (e.g., missing completely at random, missing at random, missing not at random, etc.). Before you do imputation, it is important to establish the pattern of missingness.

If you are simply trying to prevent observations from falling out of the model due to missing values, you can try what one of my old econometrics professors in grad school used to do:

Supposing missing was coded as . in your dataset,

1. Generate a new variable ("var1_recode") based on the original variable ("var1") where the missing values have been recoded to 0

2. Create a "flag for missing" variable ("flagmiss_var1") where the variable takes on the value of 1 if the value of the original variable is missing (e.g., if var1==.) and 0 otherwise (if var1 !=.)

Your dataset should look something like:

Observation     var1    flagmiss_var1     var1_recode
1               .       1                 0
2               35      0                 35
3               29      0                 29
4               .       1                 0
5               7       0                 7
6               42      0                 42
7               55      0                 55


if fit y_i = var1_recode_i + flag_var1_i --> all 7 observations will be used

if fit y_i = var1_i --> only 5 observations will be used

If you are trying to calculate the elasticity of the variable, then you will need to use some kind of imputation. The merits of multiple imputation over single imputation is discussed in Little and Rubin's (2002) Statistical analysis with missing data. I have not read this book in a while, but I remember a chapter in there on data requirements for imputation.

Reference

Little, R. J., & Rubin, D. B. (2002). Statistical analysis with missing data (2nd ed.). Hoboken, New Jersey, USA: Wiley Interscience.

1) If the probability of an observation of a variable being missing does not depend on the value of the variable (missing at random), then imputation is generally considered better than omitting cases with NA values. I agree with @marquisdecarabas that you have to establish the nature of the missingness first.

4) Having m different imputations allows you to examine the variability in estimates derived from your imputations. For example, you would calculate elasticities of the same variable for all m imputations, report the mean, and use the distribution of the m elasticities to estimate your confidence in the mean value. This is highly desirable; multiple-imputation packages handle this. As a result:

2) Having large numbers of missing values for a variable will mean that there is low confidence in estimates that involve that variable. Multiple imputation will provide a measure of that (low) confidence.

3) For categorical versus dummy variables, I recommend looking at examples provided for the multiple-imputation program that you choose.

Maybe you are coming from a Data-Mining Background and therefore are confused.

There are different usages for imputation:

1. In Statistics for survey analysis

For example in large-scale surveys, people will not respond to certain questions. This missing questions can be imputed to create a complete data set that can be analyzed with traditional analysis methods. This surveys are conducted to make inferences about population means - the values of individual cases are not interesting. Here multiple imputation is nice, because it displays the uncertainty. In general imputation for surveys aims to keep certain statistical characteristics of the data and to not influence validy of the outcome.

1. Data-Mining/ Predictive Models

Often imputation is also used as a preprocessing step in order to improve the performance of classification / regression models. Another reason is a lot of models aren't usable with NAs in the dataset. Here the target is different form surveys. Nearly everything may be valid as long as it improves the performance of the prediction model. This also leads to your question 1: For prediction, you would choose the option that in the end gives the better result