Using Multiple Imputation Techniques in Data Analysis

Question

I recently worked with two different statisticians who both suggested different strategies for dealing with imputation of missing data. For the sake of this discussion, I'll call them Statistician A and Statistician B.

Let's say that I have missing values across some cells in many columns in a dataset (all numeric and <30% missing values in any given column), and would like to replace the missing values with imputed values. For the sake of this discussion, let's assume we know the data is all missing at random. I consulted both statisticians on the best way to proceed with imputation:

I've been told by by Statistician A that a multiple imputation technique (like PMM or CART (mice package in R)) is the most robust method to use.

After completing the imputation, I completely understand the need to pool the results in regression analyses or other statistical tests to understand the contribution of each imputed dataset. And I understand that you cannot "pool" the data outside of a statistical test by averaging or stacking the data to combine all imputed datasets into a single dataset. However, if one wants to use a multiple imputation to generate a complete (single) dataset for data visualization purposes (ex. heatmaps) or more complicated techniques like PCA it seems as though the technique is no longer appropriate.

Statistician A stated that there is no perfect imputation technique and suggested using one of the imputed datasets (chosen at random) for all analyses/visualizations. I understand that this is in direct opposition to the primary benefits of multiple imputation techniques. However, I am wondering if this option is flat out incorrect and should never be used, or if this can still be utilized as a slightly better option than direct mean imputation (or other similar single imputation techniques) even though it is not technically correct? An exploration of when/if ever this might be a plausible strategy would be most helpful.

Statistician B always goes by the book and stated that data imputed via multiple imputation can only be used in analyses where the results can be pooled (linear regression). The suggestion was to drop all rows with missing data in any column. This stance is, I think, correct but somewhat limiting (and results in massive data loss with a large dataset where there are missing values across many columns). This stance is also confusing given the large number of published papers that mention the use of multiple imputation and do not ever describe pooling results or changing the method when producing visualizations, etc. And in a case where I do need to use a regression technique and I employ multiple imputation, but then would like to visualize the results, do I create a visualization with each imputed dataset or choose the values from just one?

Would someone be able to comment on how multiple imputation techniques are currently being used in scientific papers (or mis-used) and whether we should be using single imputation techniques instead? Or is this a complete misunderstanding and there is indeed a way to use multiple imputation for the tasks I've described?

I am not a statistician myself so a simplified response would be much appreciated!

Answer 1

The value of multiple imputation is that, when pooling results across imputed datasets, the uncertainty in the imputed values (which are not known but estimated) is properly captured. Without accounting for this uncertainty you will likely end up with standard errors which are too small.

For an inferential statistic like a regression coefficient, this is the reason you can't simply 'pool' the data by averaging across imputations before fitting the model. You need to account for between-imputation variance for standard errors to be valid.

It seems like some of the things you plan to present won't have associated uncertainty (visualisations, etc). I'm sure it depends on the particulars but in general it seems to me that you're probably safer to use the imputed data rather than complete cases. MI is unbiased if your data are MAR but complete case analysis might not be. So in this case why not just 'pool' your data by averaging across imputations? The fact that between-imputation variance isn't properly captured isn't so much a problem when you aren't presenting uncertainty anyway.

As a caveat, these are my personal intuitions and a genuine missing data expert may say differently!

Answer 2

I have also been thinking about the problem. I kind of lean towards Statistician B: If the results can be pooled across imputed datasets (e.g., regression coefficient, mean/median), I would use multiple imputation; if the purpose is to visualise, for example, the changes/patterns of some variables in a single dataset, I wouldn't use multiple imputation. It seems to me that for the second purpose, complete-case analysis is more reasonable. Hope someone expert could help with this question.