I would like to classify my text sample that consist of 6 mio sentences into 30 classes that correspond to the different topics. Several topics can be assigned to each sentences, so I am doing the multi-label classification task. My training sample consists of 1000-3000 sentences for each categories, together 40000 sentences. The sentences in my training sample may repeat for some topics. In my original sample there are also some sentences that does not belong to any topic/class.
I am looking for an appropriate method to do this task in python or rapidminer. At the end I want to have the percent of sentences devoted to each class/topic. I have read about one-class SVM form LibSVM in sklearn that could do the work. Besides I found also the OneVsRestClassifier in sklearn that maybe appropriate. There are also multi-output classifiers and ensemble SVM. I am not sure what is most appropriate for my model.
Besides, in case of one-class classifier, how can I test the accuracy of my classifier? I can not use a cross-validation here for my training sample.