6
$\begingroup$

Background:

I had a data set containing 212 observations with a lots of missing values. Most of the IVs and DVs are categorical (DVs are ordinal) in nature. There are 3 DVs and about 30 IVs. My intention was to run an ordinal logistic regression. A list-wise deletion keeps only 42 observations, so I decided to use hot deck imputation to fill in the missing values. I chose similar variables as the deck variables during the hot deck imputation (the deck variables should always be categorical and as far I know there should be a maximum of 5 deck variables).

Here are my queries:

1) When I imputed via hot deck once, 169 of the observations were filled in completely. If I use these imputed values for another hot deck imputation, then all 212 observations will fill in completely. But I am not sure if it is valid to use the imputed values for a further imputation. Can anyone suggest?

2) Someone suggested me (from his experience) to use the 3 DVs as the background or deck variables for imputing all the DVs and all the IVs, because that will probably facilitate my regression results. May I know your comment about it?

3) If I see almost all of the values (except from a very few) of a continuous IV are 0.10, 0.20, 0.30, 0.40, 0.50, 0.60, 0.70, 0.80, 0.90 etc. then isn't it better to impute them via hot deck rather than via EM (as the hot deck will impute a variable with it's existing values only)?

Improve this question
$\endgroup$
8
  • $\begingroup$ How many missings does a single variable have in your data? If more than, say, 20% of cases are missing in a variable, any imputation is a bad and helpless idea. $\endgroup$
    – ttnphns
    Commented Jan 28, 2013 at 8:19
  • $\begingroup$ The four variables with highest number of missing values have 62, 58, 41 and 41 missing respectively (out of 212). After imputation has been done once 20, 7, 8 and 3 values of the corresponding variables still remain missing. I have seen that the analysis cannot be done with these many missing values. What should I do now? :( $\endgroup$
    – Blain Waan
    Commented Jan 28, 2013 at 9:37
  • $\begingroup$ Hmm 29% initial missing rate is indeed large. Did you use my macros for hot-deck found on my web page? By default they allow a donor to act only once, but you could permit repeated using of a donor. Another way to preclude failing imputations is to make matching on backgroud vars more liberal (by merging some categories or just allowing partial match). $\endgroup$
    – ttnphns
    Commented Jan 28, 2013 at 11:30
  • $\begingroup$ Regarding your point 3. EM imputation (as implemented, for example, in SPSS) acts recurrently, so it is superior to hot-deck. The only situation when hot-deck is possible whereas EM is not is when the variable-to-impute is categorical. Also, EM makes assumptions about shapes of distributions while hot-deck is assumption-free (which makes it more general but in no way "better") $\endgroup$
    – ttnphns
    Commented Jan 28, 2013 at 12:43
  • $\begingroup$ @Blain, frankly, it looks like you are using a poor software implementation and dubious advice. I don't see any reason why a properly coded hot deck procedure should leave any missing values unfilled. I am not sure I am even getting your point 2 on using three DVs (which I assume are dependent variables), and what the best relation of the procedure to regression analysis would be. Generally, I would advise to move on to multiple imputation, and listen to Stef van Burren's advice on it (who wrote a book or two on this). $\endgroup$
    – StasK
    Commented Jan 28, 2013 at 14:10

1 Answer 1

Reset to default
6
$\begingroup$

Hot deck is often a good idea to obtain sensible imputations as it produces imputations that are draws from the observed data. However, filling in a single value for the missing data produces standard errors and P values that are too low. For correct statistical inference could use multiple imputation. It is easy to apply hot deck imputation in combination with multiple imputation. The most popular technique for doing this is known as predictive mean matching, and has been implemented on a variety of platforms.

Improve this answer
$\endgroup$
4
  • $\begingroup$ filling in a single value for the missing can you explain this, - i.e. the way hot-deck works to you? $\endgroup$
    – ttnphns
    Commented Jan 28, 2013 at 8:16
  • $\begingroup$ Replacing each missing entry by one data value is seductive, but current complete-data software that reads the imputed data does not distinguish between the real and the artificial data points. Instead it treats everything as real. Consequently, the software "thinks" that there is more information than there is in reality, and produces too small standard errors and uncorrect statistical tests. These problems of single imputation have been well documented in the book of Little and Rubin (2002). $\endgroup$
    – Stef van Buuren
    Commented Jan 28, 2013 at 9:16
  • $\begingroup$ In hot deck the missing values are not imputed by one data value, but with different values of the variable that is intended to impute. The values used for imputation are from within the real values of the variable that have missing values. The deck variables help find the match for the missing values. I am still not clear with solution of the three points I mentioned. If someone gives his answers point-wise, that will be really a kind help. $\endgroup$
    – Blain Waan
    Commented Jan 28, 2013 at 9:44
  • $\begingroup$ @Stef, your point that any single imputation is inherently limited because it doesn't recognize the difference between the real gathered data and the guess imputed values is absolutely right. Still, single imputation with either noise or random selection is not so bad. In many cases (e.g. in exploratory rather than inferential alalysis) to use multiple imputation is cumbersome and hot-deck or EM remain nice alternatives. $\endgroup$
    – ttnphns
    Commented Jan 28, 2013 at 11:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.