Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.

I have 60,000 data and around 45% of them is missing and the missing values are random. Can I simply use listwise or pairwise deletion or do I have to use imputation? If imputation is recommended which imputation is the best one?

share|improve this question
I'm curious what precisely you mean by "missing values are random" and how you know it. I recommend Andrew Gelman's chapter on imputation – Michael Bishop Nov 3 '12 at 23:06

2 Answers

up vote 4 down vote accepted

It depends on

  1. Amount of missing data (what percentage of data is missing)
  2. Type of missing data (MAR, MCAR, NMAR)

According to this nice article, if more than 10% data is missing, the best solution is

  1. Maximum likelihood imputation if data are NMAR (non-missing at random)
  2. Maximum likelihood and hot-deck if data are MAR (missing at random)
  3. Pairwise deletion, hot-deck or regression if data are MCAR (missing completely at random)

Here is link to that article, if it cannot be downloaded from sciencedirect.

share|improve this answer
Looks like an interesting article, thanks for the link. – Peter Flom Oct 9 '12 at 10:16
Thanks for your reply and interesting article. – Bensor Beny Oct 9 '12 at 10:33
1  
Multiple imputation is conspicuous in its absence from the linked paper. It's not mentioned at all. – Robert Long Nov 1 '12 at 10:51

In short: If your data is missing completely at random (MCAR), i.e., a true value of a missing value has the same distribution as an observed variable and missingness cannot be predicted from any other variables, your results will be unbiased but inefficient using listwise or pairwise deletion.

Multiple imputation by chained equations is regarded the best imputation method by many researchers.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.