3
$\begingroup$

This is data cleaning and preparation stage question for me. I apologize if the question is basic, but I am a beginner. I have a dataset of a bit less than 4500 records. This is a survey and year of birth is an important field. Now 670 records do not have this information. I am inclined to think that I should treat this field as 'unknown' but I wanted to ask you a question: Does it ever make sense to impute year of birth?

Perhaps you could also point me to any readings about whether demographic data can or should be imputed? Many thanks for your thoughts.

Improve this question
$\endgroup$
3
  • 3
    $\begingroup$ Two questions. Do you have any idea about highly correlated your other variables are with year of birth? If there is little correlation then it will be hard to impute accurately, but if they are correlated you might have some success. Second question: do you have any idea about whether or not there is a pattern to the missingness? For example, are these likely to be old records, or something otherwise systemic like that? $\endgroup$
    – Deathkill14
    Commented Apr 24, 2014 at 13:04
  • $\begingroup$ Thanks for pointers. My all other demographic parameters are factors - gender, income band, education, married status, then there are 100 yes/no questions. Do you think I should convert the other demographics to numerical variables and check for correlation then? $\endgroup$
    – r0berts
    Commented Apr 24, 2014 at 13:30
  • $\begingroup$ I realize that correlation is not what I should do with categorical data, probably Chi-squared test. However I tested the main outcome variable (binary) and correlation with YOB is -0.061 $\endgroup$
    – r0berts
    Commented Apr 24, 2014 at 14:15

2 Answers 2

Reset to default
10
$\begingroup$

Whether or not it makes sense to impute year of birth, and how to do it involves a number of considerations.

Firstly, imputation is probably only reasonable if the missingness pattern is Missing Completely at Random (MCAR) or Missing at Random (MAR). Some discussion of these missingness types are given in Section 25.1 of this paper. Ask yourself which type of missingness you are likely finding yourself confronted with. If you believe that there is a mechanism to the missingness you observe you may want to reconsider imputation.

Another question is do you consider birth year a categorical or a continuous variable. If you believe it should be treated continuously, you can make use of a number of imputation methods. Multiple imputation may be one of the most appropriate. An bird's eye view is given here.

If you believe birth year should be treated categorically, you face the challenge of imputing a categorical variable. This is treated in this paper, which discusses the merits of a number of imputation procedures for categorical variables and provides some examples.

Multiple imputation for continuous and categorical variables can both be performed using the mi package in R.

So to summarize, you can impute birth year whether you want to treat it continuously or categorically. First though, think about whether there is a reason why those observations might be missing. Do you think they are MCAR or MAR, or can you imagine there is a systemic reason for their missingness? If so, is it Missingness that depends on unobserved predictors? If so, can you model the missingness somehow and prevent this from biasing you imputation? Is it Missingness that depends on the missing value itself? In either of the last two scenarios, you may want to think carefully about how to proceed with your analysis, and what conclusions you can reasonably draw from it.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thank you, this is a great answer; it gives me a good idea about my dataset and further it narrows down the information I need to look review considerably. I guess that what you mean by YOB being a continuous variable is very similar to the question whether age is thought to matter or not for the outcome. Thanks again. $\endgroup$
    – r0berts
    Commented Apr 24, 2014 at 15:15
  • $\begingroup$ Up-voted this answer. All too often MI is viewed as a panacea for missing data. It's not. I also think MI is assumed with "more credibility" in my analysis. It's not. I've found more and more in my field that the MCAR and MAR assumptions aren't often theoretically defensible. However, a place where I do use MI often is in planned-missingess designs. $\endgroup$
    – bfoste01
    Commented Aug 26, 2014 at 21:24
2
$\begingroup$

Adding on to @Deathkill14 's thorough response, take a look at the package (and stand-alone application) Amelia II as well. This is another R-based method for multiple imputation. The caveats around missingness still apply. It will handle continuous or categorically defined variables similarly well and is robust to non-normality in the independent variables. It can also be parallelized if your data are large.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.