0
$\begingroup$

I´d like to create a machine-learning classification model with caret based on MR-imaging data that I extract from a tumor. Lets assume that the tumor is composed of three different subregions (e.g. edema, necrosis, enhancement) and I extract several parameters (continuous variables) from these subregions. However, not every tumor shows necrosis so necrosis-related parameters cannot be calculated for some patients. How should I deal with this parameters so that I can include all patients (and all parameters) into my model?

  • Excluding these patients would be an option, however I´d like to avoid it since I´d loose 10% of my sample size.
  • Setting the values to NA and impute them is probably not the right solution (since the values are in not missing (in a classical sense), but simply cannot be calculated)
  • Setting the values to "0" is probably also not recommended?

So is there any good method how I should deal with this situation so that I can include all parameters from all these patients into my machine-learning model?

Improve this question
$\endgroup$
3
  • $\begingroup$ What are you trying to classify, exactly? The 3 subregions? If so, then you do not have any missing data and I fail to see the problem. $\endgroup$
    – mkt
    Commented Aug 4, 2016 at 9:35
  • $\begingroup$ No. I´d like to use all the imaging parameters to train a ML-classifier that predicts disease outcome $\endgroup$
    – user86533
    Commented Aug 4, 2016 at 11:04
  • $\begingroup$ That helps a bit. But it sounds like you have a nested structure to your data (tumour > subregions (present/absent) > subregion characteristics) which is not at all clear in your question. And the types of disease outcomes you are trying to predict (and how they relate to this nested structure) might be helpful to know as well. $\endgroup$
    – mkt
    Commented Aug 4, 2016 at 12:42

1 Answer 1

Reset to default
1
$\begingroup$

Please be skeptical of the following, as it is just something I thought up just now.

One way could be to create a simpler model for all the data, excluding the regions of missing data. The predictions from this model could be ensembled with a more complex model for the 90% of the data for which all attributes are present.

Another way, would be to create a model that could learn whether extra attributes were available or not by including an extra boolean attribute that would denote this--I'm not sure if it would work in practice, but a neural network would at least theoretically be able to approximate such a model. For other models, eg. models based on decision trees, missing data can potentially be handled intrinsically (by setting the missing data to NA or some other value than can be thresholded to signify missing).

A third way, would be creating meta data, by eg. predicting the missing values in a two step classifier.

If the problem arises from you trying to do multiclass classification, perhaps consider simplifying the problem to several binary classifications.

Also, consider whether the fact that data is missing is potentially useful information. For example, maybe the missing data is due to different data gathering techniques that could perturb the results, in which case it might be important information to know if the data is missing or not.

Improve this answer
$\endgroup$
1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.