5
$\begingroup$

I have a dataset where the features are configurations (numeric values) that describe the situation and the label (only one) is the ranking of the situation (natural value between $[1,5]$). If label is $1$ then the configurations are bad and if the label is $5$ then the configurations are great.

I'm trying to use the SVM algorithm. As I understand, the SVM algorithm is designed for binary classifiers and my problem requires a multi-class classifier. I'm trying to understand how the one-vs-one and one-vs-rest can help here. Is it possible to explain how the SVM algorithm works with those two approaches in the case of one label in range [1,5]? Does my dataset considered as multiclass dataset? What If I have five labels (ranking $[1,5]$) and each one has a binary output ($0$ or $1$)? I'm a bit confused about it.

$\endgroup$

1 Answer 1

4
$\begingroup$

Is it possible to explain how the SVM algorithm works with those two approaches in the case of one label in range [1,5]

Your target variable (the 'label') $y$ can have 5 realizations: 1,2,3,4,5. Classifying observations into these 5 categories is considered a multiclass problem, yes.

The two approaches that you mentioned, one-vs-one and one-vs-rest, aim to make a binary classifier applicable to non-binary classification problems. Note that these approaches are not limited to SVM. So what do these approaches do?

One-vs-one

In your case, $|y| \cdot \frac{|y| - 1}{2}=10$ models are trained, where |y| is 5 in your case. Each model discriminates between different realizations of $y$:

  • 1 or 2
  • 1 or 3
  • 1 or 4
  • 1 or 5
  • 2 or 3
  • ...
  • 4 or 5

One-vs-rest

A coarser approach that fits 5 models that perform a 'this or other' discrimination:

  • 1 or not 1
  • 2 or not 2
  • 3 or not 3
  • 4 or not 4
  • 5 or not 5

It is noticable, that the number of models that need to be fitted is growing quite quickly for the one-vs-one approach, while one-vs-rest always requires |y| models.

What If I have five labels (ranking [1,5]) and each one has a binary output (0 or 1)? I'm a bit confused about it.

Say instead of the vector $y$ that we had above, you have a matrix $Y$ with one column per possible class realization. The values in each column are binary, 0 or 1.
Assuming that the row sums are 1 for every row, meaning that an observation is only assigned to one class, then we basically end up with the same method as with one-vs-rest. For each column in $Y$, one can fit a model that distinguishes between the respective class and 'other'.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.