# How the SVM algorithm works with one label in range?

I have a dataset where the features are configurations (numeric values) that describe the situation and the label (only one) is the ranking of the situation (natural value between $$[1,5]$$). If label is $$1$$ then the configurations are bad and if the label is $$5$$ then the configurations are great.

I'm trying to use the SVM algorithm. As I understand, the SVM algorithm is designed for binary classifiers and my problem requires a multi-class classifier. I'm trying to understand how the one-vs-one and one-vs-rest can help here. Is it possible to explain how the SVM algorithm works with those two approaches in the case of one label in range [1,5]? Does my dataset considered as multiclass dataset? What If I have five labels (ranking $$[1,5]$$) and each one has a binary output ($$0$$ or $$1$$)? I'm a bit confused about it.

## 1 Answer

Is it possible to explain how the SVM algorithm works with those two approaches in the case of one label in range [1,5]

Your target variable (the 'label') $$y$$ can have 5 realizations: 1,2,3,4,5. Classifying observations into these 5 categories is considered a multiclass problem, yes.

The two approaches that you mentioned, one-vs-one and one-vs-rest, aim to make a binary classifier applicable to non-binary classification problems. Note that these approaches are not limited to SVM. So what do these approaches do?

## One-vs-one

In your case, $$|y| \cdot \frac{|y| - 1}{2}=10$$ models are trained, where |y| is 5 in your case. Each model discriminates between different realizations of $$y$$:

• 1 or 2
• 1 or 3
• 1 or 4
• 1 or 5
• 2 or 3
• ...
• 4 or 5

## One-vs-rest

A coarser approach that fits 5 models that perform a 'this or other' discrimination:

• 1 or not 1
• 2 or not 2
• 3 or not 3
• 4 or not 4
• 5 or not 5

It is noticable, that the number of models that need to be fitted is growing quite quickly for the one-vs-one approach, while one-vs-rest always requires |y| models.

What If I have five labels (ranking [1,5]) and each one has a binary output (0 or 1)? I'm a bit confused about it.

Say instead of the vector $$y$$ that we had above, you have a matrix $$Y$$ with one column per possible class realization. The values in each column are binary, 0 or 1.
Assuming that the row sums are 1 for every row, meaning that an observation is only assigned to one class, then we basically end up with the same method as with one-vs-rest. For each column in $$Y$$, one can fit a model that distinguishes between the respective class and 'other'.