I have 60,000 data and around 45% of them is missing and the missing values are random. Can I simply use listwise or pairwise deletion or do I have to use imputation? If imputation is recommended which imputation is the best one?

  • $\begingroup$ I'm curious what precisely you mean by "missing values are random" and how you know it. I recommend Andrew Gelman's chapter on imputation $\endgroup$ – Michael Bishop Nov 3 '12 at 23:06

It depends on

  1. Amount of missing data (what percentage of data is missing)
  2. Type of missing data (MAR, MCAR, NMAR)

According to this nice article (Tsikriktsis: A review of techniques for treating missing data in OM survey research, 2005), if more than 10% data is missing, the best solution is

  1. Maximum likelihood imputation if data are NMAR (non-missing at random)
  2. Maximum likelihood and hot-deck if data are MAR (missing at random)
  3. Pairwise deletion, hot-deck or regression if data are MCAR (missing completely at random)
  • $\begingroup$ Looks like an interesting article, thanks for the link. $\endgroup$ – Peter Flom Oct 9 '12 at 10:16
  • 2
    $\begingroup$ Multiple imputation is conspicuous in its absence from the linked paper. It's not mentioned at all. $\endgroup$ – Robert Long Nov 1 '12 at 10:51
  • $\begingroup$ Hello. I'm having some difficulty opening the link to the article you shared. Would you be kind enough to post a new link or let me know the title of the article so I can manually search for it? Thank you. $\endgroup$ – Seankala Jan 23 '19 at 9:25
  • $\begingroup$ The first link should be working now. $\endgroup$ – Miroslav Sabo Jan 24 '19 at 4:11

In short: If your data is missing completely at random (MCAR), i.e., a true value of a missing value has the same distribution as an observed variable and missingness cannot be predicted from any other variables, your results will be unbiased but inefficient using listwise or pairwise deletion.

Multiple imputation by chained equations is regarded the best imputation method by many researchers.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.