I am building a multi class classification model using SVM to predict the grade for essays. What can I do to improve the result especially for class 1 and class 3? Their precision and recall are really bad.

Things I have done:
1. Split train:test set to 70:30
2. Oversample the training set using smote
3. Scale the feature by using StandardScaler from Scikit-Learn
4. Perform parameter tuning by using GridSearchCv from Scikit-Learn to get the best parameter

SVM result


First, I think your workflow makes sense but you have a problem: the number of observations is so different in all three classes. When you do your cross-validation completely random you naturally oversample the most prevalent class. Therefore you optimize the model to classify these observations correct. Intuitively, if you correctly classify a lot of the obs from class 2 your clf will achieve a decent performance. The ‚low‘ number of observations from 1 and 3 are just a ‚nice to have‘ if you classify them correctly.

One way out is sampling a similar number observations for each class or assigning weights to the classes. Lastly, in the multi-class case remember, that you are not really training a classifier for class 1, 2 or 3 with an SVM. It's either one-vs-all or all-vs-all, both options are also available in sklearn.

This is the first thing I would try and then move maybe also to other classifiers.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.