1
$\begingroup$

There are 3 classes with imbalanced number of training samples. I've got the following classification metrics:

accuracy

and the following ROC curve on the validation set:

roc

As shown in the confusion matrix on validation set, it seems all the samples of Class 2 are wrongly classified. But from the ROC curve, it seems to some extent the Class 2 are good classified with a reasonable threshold.

My question is, how to improve the classification performance on Class 2? Any comments are appreciated. Thanks!

$\endgroup$

closed as too broad by gung Apr 19 '18 at 18:29

Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • 1
    $\begingroup$ How to improve the performance is model specific. ROC only gives you feedback on your model. $\endgroup$ – SmallChess Feb 24 '17 at 1:25
  • $\begingroup$ @StudentT, thanks for comments! Is this a valid ROC curve, along with the confusion matrix? From the confusion matrix, the Class 2 are all mis-classified. $\endgroup$ – mining Feb 24 '17 at 1:31
0
$\begingroup$

try to add class weights when computing your loss function derived from your label distribution. That way you will give more emphasis on labels that are misrepresented in your dataset due to low frequency. here are some other methods as well: https://machinelearningmastery.com/tactics-to-combat-imbalanced-classes-in-your-machine-learning-dataset/ But there are plenty of forma papers as well.

$\endgroup$
  • 1
    $\begingroup$ Could you cite/link at least one such paper? – Reviewer $\endgroup$ – Jim Apr 19 '18 at 18:35
  • $\begingroup$ Hi, thanks a lot for your kind answers and suggestions! $\endgroup$ – mining Apr 19 '18 at 19:51
  • $\begingroup$ do not know by heart, google is your best friend. I had the same issue and I used Keras class weights. Can't recall sources $\endgroup$ – A.Papa Apr 19 '18 at 21:15

Not the answer you're looking for? Browse other questions tagged or ask your own question.