One-class SVM vs. OneVsRestClassifier for multi-label text classification task

Question

I would like to classify my text sample that consist of 6 mio sentences into 30 classes that correspond to the different topics. Several topics can be assigned to each sentences, so I am doing the multi-label classification task. My training sample consists of 1000-3000 sentences for each categories, together 40000 sentences. The sentences in my training sample may repeat for some topics. In my original sample there are also some sentences that does not belong to any topic/class.

I am looking for an appropriate method to do this task in python or rapidminer. At the end I want to have the percent of sentences devoted to each class/topic. I have read about one-class SVM form LibSVM in sklearn that could do the work. Besides I found also the OneVsRestClassifier in sklearn that maybe appropriate. There are also multi-output classifiers and ensemble SVM. I am not sure what is most appropriate for my model.

Besides, in case of one-class classifier, how can I test the accuracy of my classifier? I can not use a cross-validation here for my training sample.

Answer 1

Yes, I've done this very same task.

The OneVsRestClassifier in scikit-learn can be used for multi-class or multi-label classification. (when used in multi-label classification, this is the same as what's called Binary Relevance in the multi-label classification literature).

This meta classifier takes on classifier as parameter (SVM is good) and runs it once for each label then outputs multiple labels depending upon how likely each one is (but does not take into account dependencies between them).