7
$\begingroup$

Currently I have a dataset with a class distribution of 75:25.

While the 1st thought that came to my mind was to use the binary classification using logistic regression. Of course, I can upsample using SMOTE etc to make reliable predictions for minority class.

However, I had another question.

Why can't this problem be formulated as anomaly detection? Meaning, since I have too much of positives (to learn from), is it okay to consider that anything that doesn't fall with the positive examples, can it be considered as negative? So, would anomaly detection work?

q1) Is there any disadvantages to this?

q2) can help me understand why do you think classification would be the best approach for imbalanced classification? If not, why do you think anomaly detection would be the best approach?

q3) when to choose binary clf or anomaly detection?

$\endgroup$
8
  • $\begingroup$ I would encourage you instead to consider probabilistic class membership predictions. Are unbalanced datasets problematic, and (how) does oversampling (purport to) help? Classification probability threshold $\endgroup$ Commented Feb 1, 2022 at 10:13
  • $\begingroup$ @StephanKolassa - Thanks for the link. Useful. I have increased the threshold from 0.5 to 0.7. But this feels like something that I am doing based on my convenience to predict more negative samples correctly. But that threshold is just a random decision. How can I justify my threshold choice of 0.7 over 0.5? Simply, say it helped identify more negatives as negatives? Hence, I was looking at anomaly detection alogos $\endgroup$
    – The Great
    Commented Feb 1, 2022 at 10:19
  • $\begingroup$ Understand that imbalanced dataset isn't a problem with arguments like class_weight, scaled_pos_weight, boosting models etc, but how does one justify threshold choice? Any insight from your experience? $\endgroup$
    – The Great
    Commented Feb 1, 2022 at 10:21
  • $\begingroup$ Per the first link in my comment above, I don't think weighting addresses the imbalance "problem". Rather, it "addresses" problems caused by misleading evaluation metrics like accuracy, precision, recall etc., see Why is accuracy not the best measure for assessing classification models? For thresholds, I recommend setting them not to optimize some arbitrary (and, again, misleading) KPI, but based on the costs of subsequent decisions. See the second link above, and stats.stackexchange.com/… $\endgroup$ Commented Feb 1, 2022 at 10:32
  • $\begingroup$ @StephanKolassa - one quick question on this topic. Should we optimize the threshold for both train and test data? Meaning currently it is 0.5 for train and test data by default. Should I change thresholds to get the result (that I would like to see) by modifying the threshold in both train and test data? $\endgroup$
    – The Great
    Commented Feb 21, 2022 at 8:28

2 Answers 2

4
$\begingroup$

There are various methods of anomaly detection, so details depend on which one you'd use, however the general idea of anomaly detection is that there is one reference class, and that everything that deviates clearly from what goes on in the reference class is classified as anomaly. This is essentially asymmetric. If you in fact have two classes, depending on the problem there may be areas of overlap between the two classes; in some applications classes may overlap more or less strongly everywhere. Using anomaly detection, no region in data space with a good number of observations from the reference class will be classified as anomaly, so this means that in your two class problem the minority class cannot be found anywhere where the majority class is present strongly enough. On the other hand, places where there are atypical observations from the majority class may be detected as anomaly even if there is nothing of the minority class around.

So in a classification problem in which you want to treat both classes in the same manner, anomaly detection doesn't look like the right approach, even if there is imbalance, as it means that all analysis is focused on one reference class. Intuitively, this introduces even more asymmetry than the imbalance implies already, and the latter is better dealt with using appropriate loss functions as discussed in the links given by @StephanKolassa in the comments to the question.

PS: "When to choose anomaly detection"? When detecting anomalies from a reference class is really what you want, such as outliers, stray observations from unknown heterogeneous sources, or erroneous observations. Particularly anomaly detection is of interest if you have a more or less clean training sample from the reference class, but you suspect that when collecting new data without class label, some observations occur that do not belong to the reference class from sources that are not represented in the training sample.

$\endgroup$
1
  • $\begingroup$ fantastic. Thanks for your help. Upoted $\endgroup$
    – The Great
    Commented Feb 1, 2022 at 12:35
1
$\begingroup$

I will explain why anomaly detection should not be considered as classification problem by taking an example of anomaly detection using time series.

Let us say we have a time series that has some anomalies, and the anomaly points are present above and below the normal time series points. Now if we apply the clustering method(DBSCAN) to this time series, you will have a cluster something as shown below.

The red points are normal ones and the black points are anomalies

Had we considered this problem as a binary classification problem, we would get two clusters, here even though I applied the clusters (k=1) to the algorithm, you can see the anomalous points are no where seen to be forming cluster, they are scattered, this does not make sense to consider as another cluster.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.