7
$\begingroup$

I normally work more on the effect estimation/causal inference side of things, where people are pretty comfortable with multiple imputation for missing data, but right now I'm working on a project that's more in the machine learning side of things.

We're expecting to have some missing data, because it's real world medical data, which invariably does.

The inclination of some collaborators is to go with the complete case type analysis, where only subjects with full data are used, but this makes me slightly nervous, as I feel like those missing data patterns might have an impact.

Is the "best practice" for machine learning tasks to use some form of imputation? If so, should this be done before feature selection?

$\endgroup$
  • 1
    $\begingroup$ The wonderful thing about prediction is that you can just go find out what works. You only need theory if you want to know why something works. $\endgroup$ – generic_user Jul 22 '18 at 1:34
  • $\begingroup$ @generic_user or to guide your choices when there are more things to try than you have resources for =p $\endgroup$ – user20160 Jul 22 '18 at 3:48
1
$\begingroup$

The inclination of some collaborators is to go with the complete case type analysis, where only subjects with full data are used, but this makes me slightly nervous, as I feel like those missing data patterns might have an impact.

I would argue that your intuition is correct, missing data can have strong predictive power which should not be thrown away.

The question is what to do with the missing data, and here are two options (out of many)

  • Use a decision tree based algorithm which can deal with missing data. In particular it will treat missing categorical data as a category of its own. For example XGboost, Light GBM, Catboost or any other advanced tree algorithm

  • For other algorithms that can't deal with NAN (e.g. logistic regression, neural networks etc): use some form of imputation on missing data: this will depend on the shape and specifics of the distribution of data. The mean is not always the best idea, and the mode, or a percentile is sometimes better

If you are mostly interested in predictive power then I suggest using tree based algorithms which have become the norm in Kaggle competitions (with great success)

$\endgroup$
  • 1
    $\begingroup$ It's worth mentioning that in the regression case, it's perfectly reasonable to create new is_missing binary feature if you believe missingness may have predictive power. $\endgroup$ – Matthew Drury Nov 1 '18 at 0:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.