2
$\begingroup$

I'm doing an online course to learn the basics of Machine Learning. This exercise is on how to use a SVM classifier with multiple classes. While the problem is specific to question 2 from this homework, I think the error is unrelated to the question itself but rather a mistake from my part. I just don't know what it is.

Training data: amlbook.com/data/zip/features.train

Testing data: amlbook.com/data/zip/features.test

(Sorry for the unclickable links: I can't post more than one link for the moment)

As you can see from the results (recall column), only the classes 0 and 1 are mostly well predicted. Some classes are not even predicted like 5 and 8. 

Classification report for classifier SVC(C=0.01, cache_size=200, class_weight=None, coef0=1.0,
decision_function_shape=None, degree=2.0, gamma=1.0, kernel='poly',
max_iter=-1, probability=False, random_state=None, shrinking=True,
tol=0.001, verbose=False):
             precision    recall  f1-score   support

          0       0.54      0.83      0.65      1194
          1       0.93      0.96      0.95      1005
          2       0.22      0.55      0.31       731
          3       0.27      0.12      0.17       658
          4       0.12      0.02      0.04       652
          5       0.00      0.00      0.00       556
          6       0.09      0.00      0.00       664
          7       0.26      0.16      0.19       645
          8       0.00      0.00      0.00       542
          9       0.21      0.56      0.30       644

avg / total       0.32      0.40      0.33      7291


Confusion matrix:
[[987  41  65  45   2   0   0   5   0  49]
 [ 35 969   0   0   0   0   0   0   0   1]
 [ 65   3 404  48  16   0   0  48   0 147]
 [204   1 165  79  17   0   0  22   0 170]
 [ 79   9 163  11  16   0   1  81   0 292]
 [ 14   1 361  21  17   0   3  51   0  88]
 [ 38   0 282  35  22   0   1  53   0 233]
 [ 22   1 178   5  29   0   2 100   0 308]
 [298  16  89  26   1   0   2   8   0 102]
 [ 83   0 133  24  17   0   2  23   0 362]]

Here is the full code I use. I have to use a polynomial kernel with parameters degree = 2, C = 0.01, coef0 = 1.0 and gamma = 1.0. Where is the error happening in my code?

import pandas as pd
from sklearn import svm, metrics

train_df = pd.read_csv(
    filepath,
    sep = "[ ]*",
    engine = "python",
    header = None
    )
train_df.columns = ["Digit", "Intensity", "Symmetry"]
train_df["Digit"] = train_df["Digit"].astype(int)

clf = svm.SVC(
    C = 0.01,
    kernel = 'poly',
    degree = 2.0,
    gamma = 1.0,
    coef0 = 1.0
    )

X = train_df.ix[:,(1,2)].values
y = train_df.ix[:,0].values

clf.fit(X,y)

expected = y
predicted = clf.predict(X)

print("Classification report for classifier %s:\n%s\n"
      % (clf, metrics.classification_report(expected, predicted)))
print("Confusion matrix:\n%s" % metrics.confusion_matrix(expected, predicted))
Improve this question
$\endgroup$
1
  • $\begingroup$ The problem may be that your classifier optimizes a different quality measures than "classifying all classes well". (Which would not be a well-defined quality measure, anyway, since there would always need to be trade-offs between classifying "easy" vs. "hard" classes.) This answer of mine on "Why is accuracy not the best measure for assessing classification models?" may be helpful. Unfortunately, I don't have the time to dig into this question, but voted to leave open. $\endgroup$
    – Stephan Kolassa
    Commented Feb 19, 2018 at 7:31

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.