1
$\begingroup$

I have 5 classes with my data 

0 - control
1 - case1
2 - case2
3 - case3
4 - case(other cases not yet discovered)

should I consider case 4 as separate labeled data or is it logical to consider this as (0 - control) and combine the data with the control dataset and train the machine learning model?

What happens if I want to combine 4 - case(other cases not yet discovered) and 0 - control and treat the whole as 0 - control ?

Improve this question
$\endgroup$
1
  • $\begingroup$ That depends on what you want the classifier to do. If you do not know whether 0 and 4 are different cases then perhaps check if they indeed are different? Do something like PCA on the data and visualize in 2D if 0 and 4 are well-separated. $\endgroup$
    – Krrr
    Commented Sep 8, 2017 at 11:11

1 Answer 1

Reset to default
1
$\begingroup$

The suggestion by DataD'oh above will work if any potentially systematic variability associated with case4 is a major determinant of overall data variability, which can be a really dodgy assumption. I would suggest other approaches instead:

  1. Use it as it's own class. Train the model and look at the predictions and see if the "others" in case4 tend to be systematically classified in any of the categories (e.g. overall in a confusion matrix or in more detail by looking at the classifications individually). If they are classified as case4 then they would truly represent something other. If they are classified in any of the other categories, then that is also relevant information for your further inference.

  2. If they predicted all over the place, you may want to exclude them from your training and predicted back into your model to see whether they share similarities with case0-3, which may aid you in identifying functional properties of the case4 individuals.

  3. Only if you are very sure from your predictive analyses above that they are systematically classified as case0-control would I go forward with training a model where case4 are reclassified as case0.

In any of your modelling endeavours, take care to perform accurate validation (external/full split if you have enough samples; nested cross-validation otherwise; "simple" CV as a last option).

HTH HAND
Carl

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.