0
$\begingroup$

Hello fellow data scientist,

I currently reading the paper by Stekhoven & Brühlmann about MissForest. I was wondering how to deal with variables that are restricted by domain knowlege. I.e. no women can not have had prostate cancer in the past, so missing values are wanted for this item. Should I just exclude such variables (were missing values are wanted / inteded) from the MissForest imputation?

If so how can I combine these variables with the imputed datasets afterwards?

I hope this is specific enough. Thanks in advance

$\endgroup$
0
$\begingroup$

Usually it is better to first apply logical rules to fill some blanks, eventually followed by algorithmical imputation.

Take e.g. a data set about house characteristics. One column is "swimming pool" with either a 1 (yes) or a missing (no). Algorithmic imputation would set all missing to "1", destroying all information about having a pool or not.

$\endgroup$
  • $\begingroup$ Ty for your reply so you would suggest recoding the variables rather than leave them out before imputation? $\endgroup$ – PythonBeginner Jun 12 at 7:12
  • $\begingroup$ If your logical imputation just fixes data errors (men are not pregnant etc.), these variables can be used for imputing other values, yes. Similarly it is better to transform other variables before imputation (e.g. taking log income etc). So you can count your recoding as a data transformation as a preliminary step before exploiting statistical associations for further steps. $\endgroup$ – Michael M Jun 12 at 7:23
  • $\begingroup$ Thanks a lot. One last question what should I do with variables like the index number of the participant. I need them in my inputed dataset however since they just increment for every new observation they should have no predictive values. Would you recommend keeping them in my dataset? $\endgroup$ – PythonBeginner Jun 12 at 7:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.