2
$\begingroup$

There are multiple resources and answers on type of imputations and packages that can help in imputing the missing values or how to use a particular package. But there are little to no resources available on how to impute a dataset. On searching, one can find threads related to this question, but don't exactly answer them.

When should imputation take place? Let's say there is a dataset which is split into train and validation set. Should imputation take place before splitting the dataset or after splitting on training dataset? If on training dataset, how should validation split be imputed?

Follow up question. Let's say we are using KNN to impute the dataset. If I build an application that takes in a datapoint, I would need to impute the missing values in that datapoint before running predictions on it. Since KNN looks for closest rows, I feel I should add this datapoint to my imputed dataset on which prediction model was developed, run KNN again on it, get imputed datapoint and then run predictions on it. Is this correct way to do it? Would this process change for other imputation methods, like BPCA?

$\endgroup$
  • $\begingroup$ If you are looking for some background on imputation methods in general see my answer and the references here: stats.stackexchange.com/questions/257672/…. To summarize a little bit: if you assume the missing data dependent on known variables, you can try to estimate the missing value (multiple times). Do note that the required 'choices' need to be done in conjunction with the final analysis, making imputation only one of the steps in the complete process from data towards the results of the analysis. ... $\endgroup$ – IWS Aug 2 '17 at 8:43
  • $\begingroup$ cont'd with regard to the myriad of subquestions: generally, I'd say imputation would be performed before data-splitting as this allows using all data for imputation. I do not really understand what you mean with your follow-up question though. If you are asking about the data required to predict outcomes for new cases, after model development which included imputation of missing values, then yes, you would need a case with complete predictor information. $\endgroup$ – IWS Aug 2 '17 at 8:52
  • $\begingroup$ @IWS I've gone through a literature on imputation and have found that dataset I'm working on is a case of MAR. First comment of yours makes complete sense to me. Thank you for sharing your answer. For your second comment, let's take the case where I'm imputing whole dataset (using KNN, BPCA etc) and build a predictive model around it. Now if I have to use this predictive model to predict for new test datapoint, how should I impute (assuming the test datapoint has missing values)? I cannot apply KNN on this one new entry, since it does not have any neighbouring records to impute with. $\endgroup$ – Manraj Singh Aug 2 '17 at 12:38
  • $\begingroup$ AFAIK, this occurs in model validation studies (that's what they're called in the biomedical field; ncbi.nlm.nih.gov/pmc/articles/PMC3999945). In that case multiple imputation is indeed used, and based on an entire dataset of new (unseen) cases. For a single new case however, you can't. Or at least you can't base imputation values on an imputation model including this new case. What you could do is sample 100 plausible values for these missings and pool (e.g. average) all 100 resulting predictions showing the variance of these predictions as an another measure of uncertainty. $\endgroup$ – IWS Aug 2 '17 at 12:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.