# Data pre-processing: Multiple imputation

Q1: When using multiple imputation, a missing value is replaced by multiple estimated values at a time, thus creating multiple samples of the original one. If this is the case, should we run the algorithm of our choice on all samples? If so, how do we combine the final (multiple) results into one?

• Please break this into several separate questions, making each as clear as you can, & searching the site to see which have already been answered before posting any. – Scortchi - Reinstate Monica Jan 5 '15 at 14:11
• To quote from the Help Center "Your questions should be reasonably scoped. If you can imagine an entire book that answers your question, you’re asking too much." There are interesting questions in what you ask, so I'd say bring focus to the ones that are most pressing for you and follow the advice given by @Scortchi. – Graeme Walsh Jan 5 '15 at 14:16
• Also see Harrell (2001), Regression Modelling Strategies for discussion of & case studies illustrating many of these issues. – Scortchi - Reinstate Monica Jan 5 '15 at 14:46

Your questions cover a lot of ground, most likely too much for a single question. Nevertheless, I will try my best to answer them in general and point to some resources in order to help you in reformulating some of the questions as clearly as possible, as people suggested in comments.

1. What is the order in which the steps of data pre-processing should be done?

To prevent repeating myself, let me refer you to my earlier relevant answers: on data cleaning, on reproducible research and on data analysis workflows.

1. Multiple imputation

van Buuren, S., & Groothuis-Oudshoorn, K. (2011). mice: Multivariate imputation by chained equations in R. Journal of Statistical Software, 45(3). Retrieved from http://www.jstatsoft.org/v45/i03/paper.

Keep in mind that, in addition to general information, it focuses on the MICE approach, so reading additional information on alternative approaches/methods might be beneficial. For example, this vignette paper, focused on the EMB approach-based Amelia II R package: http://cran.r-project.org/web/packages/Amelia/vignettes/amelia.pdf.

1. Is "expectation-maximization" a better method for imputing missing variables?

I don't think this question makes sense, as any other question, asking about relative terms ("better") without explicitly specifying criteria for comparison or assessment. However, the description of the EM approach is available in the excellent book The Elements of Statistical Learning (section 8.5).

1. Variables redundancy

This is a rather large area and it is beyond the scope of my answer, but I will just mention that this question is, at least partially, related to the topics of feature selection, feature extraction and dimensionality reduction.

1. and 6.