# LibSVM - Multi class classification with unbalanced data

I tried to play with libsvm and 3D descriptors in order to perform object recognition. So far I have 7 categories of objects and for each category I have its number of objects (and its pourcentage) :

Category 1. 492 (14%)

Category 2. 574 (16%)

Category 3. 738 (21%)

Category4. 164 (5%)

Category5. 369 (10%)

Category6. 123 (3%)

Category7. 1025 (30%)

So I have in total 3585 objects.

I have followed the practical guide of libsvm. Here for reminder :

A. Scaling the training and the testing B. Cross validation C. Training D. Testing

I separated my data into training and testing. By doing a 5 cross validation process, I was able to determine the good C and Gamma.

However I obtained poor results (CV is about 30-40 and my accuracy is about 50%).

Then, I was thinking about my data and saw that I have some unbalanced data (categories 4 and 6 for example). I discovered that on libSVM there is an option about weight. That's why I would like now to set up the good weights.

So far I'm doing this :

svm-train -c cValue -g gValue -w1 1 -w2 1 -w3 1 -w4 2 -w5 1 -w6 2 -w7 1


However the results is the same. I'm sure that It's not the good way to do it and that's why I ask you some helps. I saw some topics on the subject but they were related to binary classification and not multiclass classification. I know that libSVM is doing "one against one" (so a binary classifier) but I don't know to handle that when I have multiple class.

• Not a bad question. But may be off topic for this site. Think about migrating to StackOverflow. – Michael Chernick Jan 11 '17 at 13:51
• "I know that libSVM is doing "one against one" (so a binary classifier) but I don't know to handle that when I have multiple class." Why are you married to using SVMs? – David Ernst Sep 15 '17 at 15:18
• You should weight the classes inversely proportional to their frequency, so that the less frequent classes get a higher weight and vice versa. Also make sure to calculate "balanced accuracy". – DataD'oh Sep 15 '17 at 16:42

You dont need to do anything special to work with multiclass problem in LibSVM. Just give the proper label to each instance (1, 2, ..., n).

Internally, LibSVM will perform a "one against one" problem for each two class. It means that for each two class, an SVM will be trained.

The probs matrix for any new prediction will be of size M = (N (N-1)) / 2, e.g, if you have 7 classes N=7, M = 21 SVM models will be created.

Please, keep in mind that Libsvm won't respect the order of your class labels, i.e, the order of the probs matrix comparison depends on the appearance order of your class labels during the training:

• "Internally class labels are ordered by their first occurrence in the training set. For a k-class data, internally labels are 0, ..., k-1, and each two-class SVM considers pair (i, j) with i < j. Then class i is treated as positive (+1) and j as negative (-1). For example, if the data set has labels +5/+10 and +10 appears first, then internally the +5 versus +10 SVM problem has +10 as positive (+1) and +5 as negative (-1)." http://www.csie.ntu.edu.tw/~cjlin/libsvm/faq.html#f430

========================================================================

In any case, check all the official responses http://www.csie.ntu.edu.tw/~cjlin/libsvm/faq

• Following the link you provide, LibSVM DOES use 'one against rest', not 1-1 ! – aginensky Apr 9 '18 at 15:01