26
$\begingroup$

I am trying to develop a predictive model using high-dimensional clinical data including laboratory values. The data space is sparse with 5k samples and 200 variables. The idea is to rank the variables using a feature selection method (IG, RF etc) and use top-ranking features for developing a predictive model.

While feature selection is going well with a Naïve Bayes approach, I am now hitting an issue in implementing a predictive model due to missing data (NA) in my variable space. Is there any machine learning algorithm that can carefully handle samples with missing data?

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ The existence of upvoted answers implies to me that this question is not too broad to be answerable. I'm voting to leave open. $\endgroup$
    – gung - Reinstate Monica
    Apr 26, 2017 at 16:20

5 Answers 5

Reset to default
16
$\begingroup$

It depends on the model you use. If you are using some generative model, then there is a principled way to deal with missing values (). For example in models like Naive Bayes or Gaussian Processes you would integrate out missing variables, and choose the best option with the remaining variables.

For discriminative models it is more elaborate, since that is not possible. There are a number of approaches. Gharamani and Jordan describe a principled approach, where missing values are treated like hidden variables, and a variant of the EM algorithm is used to estimate them. In a similar fashion, Smola et al. describe a variant of the SVM algorithm which explicitly tackles the problem.

Note that it is often recommended to substitute the missing values by the mean value of the variable. This is problematic, as described in the first paper. Sometimes, I have come across papers that do regression on the variables to estimate missing values, but I cannot say whether that applies to your case.

Improve this answer
$\endgroup$
3
  • 2
    $\begingroup$ it is often recommended to substitute the missing values by the mean value of the variable. Can you please point to the source? $\endgroup$
    – Sergey Bushmanov
    Jun 6, 2017 at 19:55
  • 1
    $\begingroup$ @juampa Why do you claim it is not possible to integrate out missing variables in discriminative models? We do this for logistic regression all the time. In fact, it can be shown to be equal to multiple imputation. $\endgroup$
    – AdamO
    Jan 29, 2018 at 15:18
  • 1
    $\begingroup$ @SergeyBushmanov I am with you in your confusion here. It is not often recommended to use (single) mean imputation because it leads to bias in some cases and anticonservative validation metrics in other cases. $\endgroup$
    – AdamO
    Jan 29, 2018 at 15:19
8
$\begingroup$

The R-package randomForestSRC, which implements Breiman's random forests, handles missing data for a wide class of analyses (regression, classification, survival, competing risk, unsupervised, multivariate).

See the following post:

Why doesn't Random Forest handle missing values in predictors?

Improve this answer
$\endgroup$
3
$\begingroup$

Try imputation using nearest neighbours to get rid of missing data.

Additionally, the Caret package has interfaces to a wide variety of algorithms and they all come with predict methods in R that can be used to predict novel data. Performance metrics can also be estimated using k-fold cross validation using the same package.

Improve this answer
$\endgroup$
3
$\begingroup$

There are also algorithms that can use the missing value as a unique and different value when building the predictive model, such as classification and regression trees. such as xgboost

Improve this answer
$\endgroup$
2
$\begingroup$

lightgbm can handle NaNs from the box(http://lightgbm.readthedocs.io/en/latest/).

Improve this answer
$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.