I have recently come to know about imputation techniques, which, in short, "guess" realistic values with which to replace missing values in a dataset. My big issue with this is that we are guessing data by assuming that they are similar to the ones we already had, which is going to reinforce any pattern that might be in the data, potentially turning a non-significant pattern into a significant one. How is this practice acceptable? What am I missing?

I am relatively new to the topic but I have done some studying and I am aware that imputation techniques range from replacing all NA with a fixed "realistic" value, to replacing it with the mean value of the observed values, to guessing the missing values with nearest-neighbor methods or with maximum likelihood methods. While I understand how these methods work I cannot shake off me the idea that they are crafting data. Imputation techniques differ in complexity and in how close to real the crafted data may look, but they are still crafting data. To me, this practice defeats the whole point of statistics as a tool to draw realistic inferences about a population based on a real, untampered sample of it, and not just a realistic sample of it. My question, to paraphrase Ian Malcolm, is not about whether we can do it but whether we should.

The first of Tukey's principles against statistician's hubris states:

The combination of some data and an aching desire for an answer does not ensure that a reasonable answer can be extracted from a given body of data.

(From "Sunset Salvo", The American Statistician 40(1), 72-76, February 1986)

Doesn't imputation collide with it?

I realise that it may just be my ignorance talking, which may be making any statistician reading this livid. If that's the case, please enlighten me. I would also appreciate pointers towards relevant literature. So far I only read the relevant chapter in Robinson's "Forest analytics in R". Cheers!

  • 2
    $\begingroup$ +1. This is precisely why I have always been extremely uncomfortable with the entire concept of imputation for missing values. I'm looking forward to learning from answers. $\endgroup$ – Stephan Kolassa Aug 19 '20 at 9:23
  • 2
    $\begingroup$ (+1) @StephanKolassa The key to it understanding this is where data are missing completely at random, or missing at random (missing data can be predicted from observed), all should be, good provided the imputation model is specified correctly. I'm talking about multiple imputation btw. The problem is that when data are missing not at random, things can fall apart pretty quickly, and there is no way to know when data are MNAR unless you "know" they are, and even when you do it's difficult to do anything about. Will try to answer in detail if I have ime later :) $\endgroup$ – Robert Long Aug 19 '20 at 9:36
  • $\begingroup$ I think you are missing the fact that sensible multiple imputation methods do not simply create a single value but a range of values ( eg dont replace by mean but by random samples with same mean and standard deviation), so that you see an uncertainty in your estimates. (and obviously if you can avoid imputation then you should, but given the choice of making some sensible estimate and no estimate I know what I would choose) $\endgroup$ – seanv507 Aug 19 '20 at 9:52
  • $\begingroup$ @seanv507: even sampling from a distribution that was learned from the valid observations will tend to reinforce our (possibly misplaced) confidence in precisely this distribution, so the critique would still stand. Regarding your choice: if you can't get a useful estimate without imputation, then I would question what you would base your imputation on - and whether any estimates based on that imputation would be of any use. $\endgroup$ – Stephan Kolassa Aug 19 '20 at 10:16
  • $\begingroup$ I really don't like imputation. I think it's just a work around to when you don't want top do proper MLE estimation with missing data $\endgroup$ – Aksakal Aug 19 '20 at 16:32

There is no clear cut answer here. The fun though is that one can verify the effects of the imputation using a validation procedure: let the data decide!

Should one throw away a feature if a few values are missing? Or the observations then? What if those observations have valuable information in the other features and your algorithm cannot handle missing values? And so on.

Imputation, like removing observations or features, is just a way of dealing with missing values. The decison of which one is best should be supported by good machine procedures like (cross-)validation.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.