Using confusion matrix to improve my SVM

I ran an SVM classifier on the CIFAR_10 classification workbench. I got about 2/3 accuracy (which I think is Ok, but I want to improve...)

Here is my confusion matrix:

[[719  12  70  26  37  12  14   6  72  32]
[ 14 783  13  21  11   8  20   5  40  85]
[ 69  15 551  92  97  58  58  27  19  14]
[ 27  14  87 502  89 148  57  33   9  34]
[ 29  11  95  82 590  56  48  57  17  15]
[ 14   8  91 185  57 531  45  45   9  15]
[ 21  25  48  76  37  39 715  13  11  15]
[ 12   8  41  62  79  53  11 701   8  25]
[ 72  56  22  25  17   7  11   6 759  25]
[ 44  83  16  30  19  20  11  26  29 722]]

• I noted that the 4th and 6th classes (counting from index 1) have much in common, meaning the error rate between those two is high ( (148,185) images wrongly classified).
• I also noted the 3rd to 6th classes are classified in a less precise way than the others.

My SVM (polynomial kernel) was trained on 50K images using SIFT descriptors, and was tested on 10K.

My question is - how can I use the facts I stated to improve the classifier? I want to somehow use the confusion matrix to train it again, with more attention to the differences between those (3rd to 6th) classes. I have no objections to using other classifiers, but as an improvement to existing results, not a replacement.

So, can I re-fit the classifier with specific data that suits my needs?

• Perhaps change the cost function to value the 3rd and 6th more? – a13a22 Jan 1 '18 at 17:12
• Thanks. This is all very new to me, and I'm using Python 2 sklearn.svm SVC() so, can you explain how can I determine a different cost for different classes? – CIsForCookies Jan 1 '18 at 17:24
• Take a look here: stackoverflow.com/q/25070910/5455789 – a13a22 Jan 1 '18 at 18:05
• Thanks! now, what should be heavier? Both the confusing classes? only the "good" classes? How does this work? – CIsForCookies Jan 1 '18 at 19:32
• You want to allocate a higher cost of misclassification on those bad classes so that the system is forced to conform with that they perform better – a13a22 Jan 3 '18 at 16:36