6
$\begingroup$

I have a binary classification problem, where my training data is 70% positive labeled and 30% negative labelled. I use a logistic loss and it always classifies examples positive on the test data.

How can I make it classify some examples negative as well? One solution I though of might be to remove some positive examples in the training data so that it's 50% 50% pos neg labels. Another is to use a non-linear classifier because the problem might be non-linear.

$\endgroup$
2

1 Answer 1

6
$\begingroup$

If it is only 70%-30% there is probably no need to balance the dataset. The class imbalance problem is caused by not having enough patterns for the minority class, rather than a high ratio of positive to negative patterns. Generally, if you have enough data, the "class imbalance problem" doesn't arise. Also, note that if you artificially balance the dataset, you are implying an equal prior probability of positive and negative patterns. If that isn't true, your model may give bad predictions by over-predicting the minority class.

More importantly, there may be an overlap between classes such that the Bayes optimal decision is always to assign patterns to the positive class, in which case your model is doing exactly the right thing. Consider the case where there is one explanatory variable, which is distributed according to a standard normal distribution for both classes. In that case, as the positive class has a higher prior probability, the optimal model assigns all patterns to the positive class. Similar examples can be constructed where the class means are not the same, but the difference is small compared with the variation.

If classifying the majority class is a problem, that suggests that the misclassification costs of false-positive and false-negative costs are not the same. This can be built into the classifier by changing the threshold, rather than the model, as you are using a logistic loss.

$\endgroup$
4
  • $\begingroup$ so the problem is that the model doesn't do better than a baseline of classifying everything positive blindly. Actually, my loss is defined as prediction * label where prediction ranges -1 - +1 and label can be any number. $\endgroup$
    – siamii
    May 18, 2013 at 13:48
  • $\begingroup$ Also, misclassifying a positive as negative has a much larger cost than misclassifying negative as positive, because label tends to be large positive numbers, but sometimes it is small negative number. $\endgroup$
    – siamii
    May 18, 2013 at 14:23
  • $\begingroup$ I'm sorry, I don't really understand the loss defined in your earlier comment (it doesn't look like the logistic loss). $\endgroup$ May 18, 2013 at 14:26
  • $\begingroup$ So the point of the loss is that that algorithm should correctly classify negative and positive labels. But if the label is really large, it has more significance. So misclassifying a label +100 as negative has more cost than a label +1. $\endgroup$
    – siamii
    May 18, 2013 at 15:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.