3
$\begingroup$

When we have two classes A, B, can we use one classifier which will be trained on class A samples and tested on both class A, class B samples?

When to use binary classifier and one-class classifier?

Can I use one-class classifier for any type of binary class data?

  • Example 1: Faulty and non-faulty machinery data. Can I implement one-class classifier?

  • Example 2: Positive and negative sentiment text data. Can I implement one-class classifier?

$\endgroup$
2
6
$\begingroup$

Suppose you are trying to perform two-class classification on faulty and non-faulty machinery data, where each example in the dataset is represented using the feature vector $\mathbf{x} = [x_1 \ x_2]^T$. This could be done as shown below:

enter image description here

Here, the faulty machinery data is represented by the orange area, and the non-faulty machinery data represented by the blue area. Suppose that the machines in question are rarely ever faulty, such that there are so many different examples of non-faulty machinery, but very few examples of faulty machinery. Given that you have trained on the data shown above, it is possible that you observe an example of non-faulty machinery that you mis-classify as faulty:

enter image description here

Again, the reason that this happens is because the set of all possible feature vectors representing non-faulty machinery is just too big. It is not possible to capture all of them and train on them. You could argue that you just need to collect more data and train on that, but what you would end up doing is this:

enter image description here

This, by the way, can be done using a neural network, for example, and the act of drawing these lines is the basic idea behind discriminative modelling.

However, why bother collecting so much data and creating a very complex model that can draw all of these lines, when you can just try to draw the shape of the faulty data like this?

enter image description here

In the figure above, the orange area is the faulty machinery data that you collected, while the rest of the feature space is assumed to represent non-faulty machinery. This is the basic idea behind generative modelling, where, instead of trying to draw a line that splits the feature space, you instead try to estimate the disrtribution of the faulty data to learn what it looks like. Then, given a new test vector $\mathbf{x}$, all you need to do is to measure the distance between the center of the distribution of the faulty data and this new test vector $\mathbf{x}$. If this distance is greater than a specific threshold, then the test vector is classified as non-faulty. Otherwise, it is classified as faulty. This is also one way of performing anomaly detection.

$\endgroup$
7
  • $\begingroup$ Thanks for your explanation! Consider the second example of text data. can we train a model only on positive sentiment sentences to classify samples of positive and negative sentiment. Since we have a large number of samples for both positive and negative sentiment. $\endgroup$ – SS Varshini Jan 18 at 11:54
  • 1
    $\begingroup$ That depends. How many possible sentences are there for each class? If there is a lot of examples for one class compared to the other class, then consider it as an anomaly detection task $\endgroup$ – mhdadk Jan 18 at 11:57
  • 2
    $\begingroup$ Not really. If one class is very specific, while another class is very general, then one-class classification is the way to go. For example, a faulty machine is a very specific example, but a non-faulty one is not. This is because the machine can be doing many different things while it is not faulty. Another example, in the case of audio, is if I want to detect if someone said a specific word or not. I will be scanning audio signals for a long time looking for the word, and I will have to classify all kinds of different audio as a negative... $\endgroup$ – mhdadk Jan 18 at 17:02
  • 1
    $\begingroup$ Basically, if one class is much more diverse than the other class, one-class classification is a good idea. However, if you only want to classify cats and dogs, where both classes are almost equally diverse, then two-class classification is a better solution. $\endgroup$ – mhdadk Jan 18 at 17:03
  • 1
    $\begingroup$ Ok I got it thanks for your explanation @mhdadk $\endgroup$ – SS Varshini Jan 18 at 17:43
4
$\begingroup$

You could, but why would you? There is much more to gain by training a model on both classes, so that your model can learn from both classes and better distinguish between them. Imagine trying to learn to distinguish between cats and dogs, would it help you to see only images of cats, and then try to guess which image is a dog, without ever having seen a dog?

Using a one-class classifier would make sense only if you do not have any examples from the other class at this particular point in time. Or if the "other" class could be composed of many unknown classes, which cannot be easily grouped into one official class. For ex. when doing anomaly detection, when you have only a few very weird cases, which could be the causes of many very varied reasons.

$\endgroup$
1
  • 2
    $\begingroup$ Even if they have access to both sets of examples, one-class classification could still be helpful. This occurs when the size of the set of all possible examples from one class greatly exceeds the size of the set of all possible examples from the other class. $\endgroup$ – mhdadk Jan 18 at 9:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.