I have done some surfing to find an answer but I am still stuck. I have a dataset with n rows and p columns. I have several missing data across most of the variables going from 2% up to 70%. I am aware of several techniques to impute them, but I cannot figure out what is the amount of missing data per variable for which it is safe to run the imputation, with whatever algorithm.

For instance, I found this post on r-bloggers which is basically a tutorial for the well known R's package mice. When introducing the concept of Missing At Random (MAR) and Missing Completely At Random (MCAR), the author says:

Assuming data is MCAR, too much missing data can be a problem too. Usually a safe maximum threshold is 5% of the total for large datasets. If missing data for a certain feature or sample is more than 5% then you probably should leave that feature or sample out.

Now, I am assuming my data to be MAR, so it is slightly different from the case above, but still the question holds. If I trust the guideline here I can forget to impute most of my variables.

Do you have any hints or suggestions whatsoever?

Thank you in advance,

  • 1
    $\begingroup$ You may get more helpful answers if you can give a bit more context to your problem? What's reasonable or not reasonable with regards to missing data can be problem specific. $\endgroup$ – Matthew Gunn Sep 22 '16 at 14:39
  • $\begingroup$ When in doubt, conduct simulations for your specific type of data. $\endgroup$ – Roland Sep 23 '16 at 7:12
  • $\begingroup$ @MatthewGunn I am working with clinicians on data regarding different parameters like functionals, bioelectric impedance, blood measurements and so forth. The sample is composed of patients affected by Chronic Heart Failure and we have measures for 166 units. The problem is that for several quantities, like the result of a walking test, the proportion of missing data is around 70%. As I said before, I am into missing imputation techniques and algorithms, but what I am not getting here is how far I can push. $\endgroup$ – contefranz Sep 24 '16 at 13:31
  • $\begingroup$ @Roland I iterated the analysis several times with different scenarios and what I get is not very encouraging. As expected, I do not obtain a very good agreement between empirical distribution and imputed ones which varies at every simulation iteration. I also tried to not impute those variables with significant amount of missing data, but this does not seem to affect the results on the remaining imputed variables. Any thoughts? $\endgroup$ – contefranz Sep 24 '16 at 13:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.