0
$\begingroup$

I have an extremely unbalanced data set: around 200 positive samples and 70,000 negative samples. To overcome this problem I have tried to over-sample the minority class as suggested in previous questions here.

Since training the classifier (in my case I am using SVM) takes a long time on my computer I first trained a classifier on a small subset of my data picked randomly, the classifier gave solid results when tested on the test set, yet when I trained a classifier using the full training data the classifier produced very poor results. Does anyone have insight into why this is happening and what should i do? I have also tried using weights but it did not produce a different result.

$\endgroup$
1
  • $\begingroup$ How do you set the value of the SVM hyperparameter(s); e.g. $C$ or $\lambda$? When you say a random subset of the training data do you mean 200 positive samples + a small random subset of the negative data? $\endgroup$
    – Sobi
    Dec 9, 2015 at 20:46

1 Answer 1

0
$\begingroup$

You can try F1 score that is specifically designed for skewed classes. The idea is to measure both precision and recall, because usual error ratio doesn't work for skewed classes. E. g. in your case you could always predict "negative" and yet have 99.7% correct answers.

$$F_1 = 2\frac {precision * recall}{precision + recall}$$

More info on wiki: F1 score

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.